2023-05-05 15:53:35,782 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-05-05 15:53:36,055 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930 2023-05-05 15:53:36,072 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-05-05 15:53:36,073 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/cluster_ed487d35-94af-afba-80b3-40ed4e353473, deleteOnExit=true 2023-05-05 15:53:36,073 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-05-05 15:53:36,073 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/test.cache.data in system properties and HBase conf 2023-05-05 15:53:36,074 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/hadoop.tmp.dir in system properties and HBase conf 2023-05-05 15:53:36,075 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/hadoop.log.dir in system properties and HBase conf 2023-05-05 15:53:36,075 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/mapreduce.cluster.local.dir in system properties and HBase conf 2023-05-05 15:53:36,076 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-05-05 15:53:36,076 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-05-05 15:53:36,197 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-05-05 15:53:36,655 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-05-05 15:53:36,660 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-05-05 15:53:36,661 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-05-05 15:53:36,661 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-05-05 15:53:36,662 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-05-05 15:53:36,662 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-05-05 15:53:36,663 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-05-05 15:53:36,663 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-05-05 15:53:36,664 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/dfs.journalnode.edits.dir in system properties and HBase conf 2023-05-05 15:53:36,664 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-05-05 15:53:36,665 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/nfs.dump.dir in system properties and HBase conf 2023-05-05 15:53:36,665 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/java.io.tmpdir in system properties and HBase conf 2023-05-05 15:53:36,665 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/dfs.journalnode.edits.dir in system properties and HBase conf 2023-05-05 15:53:36,666 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-05-05 15:53:36,666 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-05-05 15:53:37,346 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-05-05 15:53:37,351 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-05-05 15:53:37,701 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-05-05 15:53:37,885 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-05-05 15:53:37,906 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-05-05 15:53:37,949 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-05-05 15:53:37,987 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/java.io.tmpdir/Jetty_localhost_46225_hdfs____.fyz5lv/webapp 2023-05-05 15:53:38,141 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46225 2023-05-05 15:53:38,161 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-05-05 15:53:38,161 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-05-05 15:53:38,717 WARN [Listener at localhost/35855] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-05-05 15:53:38,814 WARN [Listener at localhost/35855] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-05-05 15:53:38,838 WARN [Listener at localhost/35855] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-05-05 15:53:38,846 INFO [Listener at localhost/35855] log.Slf4jLog(67): jetty-6.1.26 2023-05-05 15:53:38,853 INFO [Listener at localhost/35855] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/java.io.tmpdir/Jetty_localhost_42017_datanode____oyfkzj/webapp 2023-05-05 15:53:38,987 INFO [Listener at localhost/35855] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42017 2023-05-05 15:53:39,336 WARN [Listener at localhost/34563] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-05-05 15:53:39,357 WARN [Listener at localhost/34563] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-05-05 15:53:39,366 WARN [Listener at localhost/34563] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-05-05 15:53:39,368 INFO [Listener at localhost/34563] log.Slf4jLog(67): jetty-6.1.26 2023-05-05 15:53:39,377 INFO [Listener at localhost/34563] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/java.io.tmpdir/Jetty_localhost_42559_datanode____zfr2ki/webapp 2023-05-05 15:53:39,490 INFO [Listener at localhost/34563] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42559 2023-05-05 15:53:39,508 WARN [Listener at localhost/44159] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-05-05 15:53:39,544 WARN [Listener at localhost/44159] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-05-05 15:53:39,549 WARN [Listener at localhost/44159] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-05-05 15:53:39,552 INFO [Listener at localhost/44159] log.Slf4jLog(67): jetty-6.1.26 2023-05-05 15:53:39,569 INFO [Listener at localhost/44159] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/java.io.tmpdir/Jetty_localhost_39875_datanode____vfqcbb/webapp 2023-05-05 15:53:39,756 INFO [Listener at localhost/44159] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39875 2023-05-05 15:53:39,795 WARN [Listener at localhost/35117] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-05-05 15:53:39,999 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x8032aa0656b54138: Processing first storage report for DS-5f1bb721-5edc-405a-aaad-54574a665d94 from datanode 8e12ece6-ff16-4724-9684-7233909ecb4c 2023-05-05 15:53:40,001 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x8032aa0656b54138: from storage DS-5f1bb721-5edc-405a-aaad-54574a665d94 node DatanodeRegistration(127.0.0.1:35557, datanodeUuid=8e12ece6-ff16-4724-9684-7233909ecb4c, infoPort=34505, infoSecurePort=0, ipcPort=44159, storageInfo=lv=-57;cid=testClusterID;nsid=1209721193;c=1683302017433), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-05-05 15:53:40,001 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc21bcfcfc1119534: Processing first storage report for DS-3097abad-0a2f-49cd-8a8a-8998355904cb from datanode 7884a1c8-f94e-4e8e-bd24-af43573a93c6 2023-05-05 15:53:40,001 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc21bcfcfc1119534: from storage DS-3097abad-0a2f-49cd-8a8a-8998355904cb node DatanodeRegistration(127.0.0.1:36035, datanodeUuid=7884a1c8-f94e-4e8e-bd24-af43573a93c6, infoPort=44127, infoSecurePort=0, ipcPort=34563, storageInfo=lv=-57;cid=testClusterID;nsid=1209721193;c=1683302017433), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-05-05 15:53:40,001 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x8032aa0656b54138: Processing first storage report for DS-fc341830-76d2-4d17-97f5-449503635e07 from datanode 8e12ece6-ff16-4724-9684-7233909ecb4c 2023-05-05 15:53:40,001 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x8032aa0656b54138: from storage DS-fc341830-76d2-4d17-97f5-449503635e07 node DatanodeRegistration(127.0.0.1:35557, datanodeUuid=8e12ece6-ff16-4724-9684-7233909ecb4c, infoPort=34505, infoSecurePort=0, ipcPort=44159, storageInfo=lv=-57;cid=testClusterID;nsid=1209721193;c=1683302017433), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-05-05 15:53:40,002 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc21bcfcfc1119534: Processing first storage report for DS-880d1559-dd1d-4c15-a1e3-88fdedb2c82a from datanode 7884a1c8-f94e-4e8e-bd24-af43573a93c6 2023-05-05 15:53:40,002 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc21bcfcfc1119534: from storage DS-880d1559-dd1d-4c15-a1e3-88fdedb2c82a node DatanodeRegistration(127.0.0.1:36035, datanodeUuid=7884a1c8-f94e-4e8e-bd24-af43573a93c6, infoPort=44127, infoSecurePort=0, ipcPort=34563, storageInfo=lv=-57;cid=testClusterID;nsid=1209721193;c=1683302017433), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-05-05 15:53:40,766 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xade94fc3f660423c: Processing first storage report for DS-fffa5dcb-dda3-4efc-84b4-1fd59220aa20 from datanode 7b0dd175-f33e-42f4-a043-9cbab024e73c 2023-05-05 15:53:40,766 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xade94fc3f660423c: from storage DS-fffa5dcb-dda3-4efc-84b4-1fd59220aa20 node DatanodeRegistration(127.0.0.1:40251, datanodeUuid=7b0dd175-f33e-42f4-a043-9cbab024e73c, infoPort=42777, infoSecurePort=0, ipcPort=35117, storageInfo=lv=-57;cid=testClusterID;nsid=1209721193;c=1683302017433), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-05-05 15:53:40,766 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xade94fc3f660423c: Processing first storage report for DS-9747a0db-94bc-4767-bd9d-d0272ebfe8ca from datanode 7b0dd175-f33e-42f4-a043-9cbab024e73c 2023-05-05 15:53:40,767 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xade94fc3f660423c: from storage DS-9747a0db-94bc-4767-bd9d-d0272ebfe8ca node DatanodeRegistration(127.0.0.1:40251, datanodeUuid=7b0dd175-f33e-42f4-a043-9cbab024e73c, infoPort=42777, infoSecurePort=0, ipcPort=35117, storageInfo=lv=-57;cid=testClusterID;nsid=1209721193;c=1683302017433), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-05-05 15:53:40,821 DEBUG [Listener at localhost/35117] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930 2023-05-05 15:53:40,885 INFO [Listener at localhost/35117] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/cluster_ed487d35-94af-afba-80b3-40ed4e353473/zookeeper_0, clientPort=51118, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/cluster_ed487d35-94af-afba-80b3-40ed4e353473/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/cluster_ed487d35-94af-afba-80b3-40ed4e353473/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-05-05 15:53:40,899 INFO [Listener at localhost/35117] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=51118 2023-05-05 15:53:40,907 INFO [Listener at localhost/35117] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-05 15:53:40,909 INFO [Listener at localhost/35117] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-05 15:53:41,613 INFO [Listener at localhost/35117] util.FSUtils(471): Created version file at hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27 with version=8 2023-05-05 15:53:41,613 INFO [Listener at localhost/35117] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/hbase-staging 2023-05-05 15:53:41,951 INFO [Listener at localhost/35117] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-05-05 15:53:42,390 INFO [Listener at localhost/35117] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-05-05 15:53:42,422 INFO [Listener at localhost/35117] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-05 15:53:42,423 INFO [Listener at localhost/35117] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-05-05 15:53:42,423 INFO [Listener at localhost/35117] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-05-05 15:53:42,424 INFO [Listener at localhost/35117] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-05 15:53:42,424 INFO [Listener at localhost/35117] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-05-05 15:53:42,562 INFO [Listener at localhost/35117] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-05-05 15:53:42,635 DEBUG [Listener at localhost/35117] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-05-05 15:53:42,727 INFO [Listener at localhost/35117] ipc.NettyRpcServer(120): Bind to /172.31.14.131:44213 2023-05-05 15:53:42,737 INFO [Listener at localhost/35117] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-05 15:53:42,739 INFO [Listener at localhost/35117] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-05 15:53:42,767 INFO [Listener at localhost/35117] zookeeper.RecoverableZooKeeper(93): Process identifier=master:44213 connecting to ZooKeeper ensemble=127.0.0.1:51118 2023-05-05 15:53:42,806 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:442130x0, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-05-05 15:53:42,808 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:44213-0x101ef8660cb0000 connected 2023-05-05 15:53:42,834 DEBUG [Listener at localhost/35117] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-05-05 15:53:42,835 DEBUG [Listener at localhost/35117] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-05 15:53:42,838 DEBUG [Listener at localhost/35117] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-05-05 15:53:42,847 DEBUG [Listener at localhost/35117] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44213 2023-05-05 15:53:42,848 DEBUG [Listener at localhost/35117] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44213 2023-05-05 15:53:42,849 DEBUG [Listener at localhost/35117] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44213 2023-05-05 15:53:42,849 DEBUG [Listener at localhost/35117] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44213 2023-05-05 15:53:42,849 DEBUG [Listener at localhost/35117] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44213 2023-05-05 15:53:42,855 INFO [Listener at localhost/35117] master.HMaster(444): hbase.rootdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27, hbase.cluster.distributed=false 2023-05-05 15:53:42,923 INFO [Listener at localhost/35117] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-05-05 15:53:42,924 INFO [Listener at localhost/35117] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-05 15:53:42,924 INFO [Listener at localhost/35117] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-05-05 15:53:42,924 INFO [Listener at localhost/35117] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-05-05 15:53:42,924 INFO [Listener at localhost/35117] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-05 15:53:42,924 INFO [Listener at localhost/35117] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-05-05 15:53:42,929 INFO [Listener at localhost/35117] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-05-05 15:53:42,931 INFO [Listener at localhost/35117] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43023 2023-05-05 15:53:42,934 INFO [Listener at localhost/35117] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-05-05 15:53:42,940 DEBUG [Listener at localhost/35117] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-05-05 15:53:42,941 INFO [Listener at localhost/35117] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-05 15:53:42,943 INFO [Listener at localhost/35117] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-05 15:53:42,944 INFO [Listener at localhost/35117] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43023 connecting to ZooKeeper ensemble=127.0.0.1:51118 2023-05-05 15:53:42,950 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:430230x0, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-05-05 15:53:42,951 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:43023-0x101ef8660cb0001 connected 2023-05-05 15:53:42,951 DEBUG [Listener at localhost/35117] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-05-05 15:53:42,953 DEBUG [Listener at localhost/35117] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-05 15:53:42,954 DEBUG [Listener at localhost/35117] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-05-05 15:53:42,956 DEBUG [Listener at localhost/35117] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43023 2023-05-05 15:53:42,959 DEBUG [Listener at localhost/35117] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43023 2023-05-05 15:53:42,959 DEBUG [Listener at localhost/35117] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43023 2023-05-05 15:53:42,964 DEBUG [Listener at localhost/35117] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43023 2023-05-05 15:53:42,964 DEBUG [Listener at localhost/35117] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43023 2023-05-05 15:53:42,977 INFO [Listener at localhost/35117] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-05-05 15:53:42,977 INFO [Listener at localhost/35117] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-05 15:53:42,978 INFO [Listener at localhost/35117] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-05-05 15:53:42,978 INFO [Listener at localhost/35117] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-05-05 15:53:42,978 INFO [Listener at localhost/35117] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-05 15:53:42,978 INFO [Listener at localhost/35117] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-05-05 15:53:42,978 INFO [Listener at localhost/35117] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-05-05 15:53:42,980 INFO [Listener at localhost/35117] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42799 2023-05-05 15:53:42,980 INFO [Listener at localhost/35117] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-05-05 15:53:42,983 DEBUG [Listener at localhost/35117] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-05-05 15:53:42,984 INFO [Listener at localhost/35117] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-05 15:53:42,986 INFO [Listener at localhost/35117] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-05 15:53:42,989 INFO [Listener at localhost/35117] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42799 connecting to ZooKeeper ensemble=127.0.0.1:51118 2023-05-05 15:53:42,993 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:427990x0, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-05-05 15:53:42,994 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:42799-0x101ef8660cb0002 connected 2023-05-05 15:53:42,994 DEBUG [Listener at localhost/35117] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-05-05 15:53:42,995 DEBUG [Listener at localhost/35117] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-05 15:53:42,996 DEBUG [Listener at localhost/35117] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-05-05 15:53:42,999 DEBUG [Listener at localhost/35117] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42799 2023-05-05 15:53:42,999 DEBUG [Listener at localhost/35117] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42799 2023-05-05 15:53:43,000 DEBUG [Listener at localhost/35117] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42799 2023-05-05 15:53:43,004 DEBUG [Listener at localhost/35117] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42799 2023-05-05 15:53:43,004 DEBUG [Listener at localhost/35117] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42799 2023-05-05 15:53:43,017 INFO [Listener at localhost/35117] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-05-05 15:53:43,017 INFO [Listener at localhost/35117] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-05 15:53:43,017 INFO [Listener at localhost/35117] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-05-05 15:53:43,017 INFO [Listener at localhost/35117] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-05-05 15:53:43,017 INFO [Listener at localhost/35117] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-05 15:53:43,017 INFO [Listener at localhost/35117] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-05-05 15:53:43,017 INFO [Listener at localhost/35117] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-05-05 15:53:43,019 INFO [Listener at localhost/35117] ipc.NettyRpcServer(120): Bind to /172.31.14.131:40991 2023-05-05 15:53:43,019 INFO [Listener at localhost/35117] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-05-05 15:53:43,020 DEBUG [Listener at localhost/35117] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-05-05 15:53:43,021 INFO [Listener at localhost/35117] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-05 15:53:43,023 INFO [Listener at localhost/35117] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-05 15:53:43,024 INFO [Listener at localhost/35117] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:40991 connecting to ZooKeeper ensemble=127.0.0.1:51118 2023-05-05 15:53:43,033 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:409910x0, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-05-05 15:53:43,034 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:40991-0x101ef8660cb0003 connected 2023-05-05 15:53:43,034 DEBUG [Listener at localhost/35117] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-05-05 15:53:43,035 DEBUG [Listener at localhost/35117] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-05 15:53:43,036 DEBUG [Listener at localhost/35117] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-05-05 15:53:43,037 DEBUG [Listener at localhost/35117] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40991 2023-05-05 15:53:43,037 DEBUG [Listener at localhost/35117] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40991 2023-05-05 15:53:43,038 DEBUG [Listener at localhost/35117] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40991 2023-05-05 15:53:43,038 DEBUG [Listener at localhost/35117] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40991 2023-05-05 15:53:43,040 DEBUG [Listener at localhost/35117] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40991 2023-05-05 15:53:43,044 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,44213,1683302021756 2023-05-05 15:53:43,053 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-05-05 15:53:43,054 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,44213,1683302021756 2023-05-05 15:53:43,073 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-05-05 15:53:43,073 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-05-05 15:53:43,073 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-05-05 15:53:43,073 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-05-05 15:53:43,073 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-05 15:53:43,075 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-05-05 15:53:43,077 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,44213,1683302021756 from backup master directory 2023-05-05 15:53:43,077 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-05-05 15:53:43,081 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,44213,1683302021756 2023-05-05 15:53:43,081 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-05-05 15:53:43,082 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-05-05 15:53:43,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,44213,1683302021756 2023-05-05 15:53:43,085 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-05-05 15:53:43,086 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-05-05 15:53:43,180 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/hbase.id with ID: b6fc4f2f-8698-4803-99b0-5fcbfd922edc 2023-05-05 15:53:43,219 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-05 15:53:43,235 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-05 15:53:43,282 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x5f52dad7 to 127.0.0.1:51118 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-05-05 15:53:43,314 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3afde5ab, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-05-05 15:53:43,337 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-05-05 15:53:43,339 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-05-05 15:53:43,356 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-05-05 15:53:43,356 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-05-05 15:53:43,358 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-05-05 15:53:43,362 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-05-05 15:53:43,363 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-05-05 15:53:43,396 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/MasterData/data/master/store-tmp 2023-05-05 15:53:43,430 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:53:43,431 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-05-05 15:53:43,431 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-05 15:53:43,431 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-05 15:53:43,431 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-05-05 15:53:43,431 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-05 15:53:43,431 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-05 15:53:43,431 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-05-05 15:53:43,433 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/MasterData/WALs/jenkins-hbase4.apache.org,44213,1683302021756 2023-05-05 15:53:43,453 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44213%2C1683302021756, suffix=, logDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/MasterData/WALs/jenkins-hbase4.apache.org,44213,1683302021756, archiveDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/MasterData/oldWALs, maxLogs=10 2023-05-05 15:53:43,511 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35557,DS-5f1bb721-5edc-405a-aaad-54574a665d94,DISK] 2023-05-05 15:53:43,511 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36035,DS-3097abad-0a2f-49cd-8a8a-8998355904cb,DISK] 2023-05-05 15:53:43,511 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40251,DS-fffa5dcb-dda3-4efc-84b4-1fd59220aa20,DISK] 2023-05-05 15:53:43,519 DEBUG [RS-EventLoopGroup-5-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-05-05 15:53:43,582 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/MasterData/WALs/jenkins-hbase4.apache.org,44213,1683302021756/jenkins-hbase4.apache.org%2C44213%2C1683302021756.1683302023463 2023-05-05 15:53:43,583 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40251,DS-fffa5dcb-dda3-4efc-84b4-1fd59220aa20,DISK], DatanodeInfoWithStorage[127.0.0.1:36035,DS-3097abad-0a2f-49cd-8a8a-8998355904cb,DISK], DatanodeInfoWithStorage[127.0.0.1:35557,DS-5f1bb721-5edc-405a-aaad-54574a665d94,DISK]] 2023-05-05 15:53:43,583 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-05-05 15:53:43,584 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:53:43,587 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-05-05 15:53:43,588 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-05-05 15:53:43,644 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-05-05 15:53:43,652 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-05-05 15:53:43,675 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-05-05 15:53:43,687 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:53:43,693 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-05-05 15:53:43,694 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-05-05 15:53:43,708 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-05-05 15:53:43,712 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-05 15:53:43,713 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75238721, jitterRate=0.12114430963993073}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-05-05 15:53:43,713 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-05-05 15:53:43,714 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-05-05 15:53:43,733 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:53:43,733 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:53:43,736 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:53:43,737 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-05-05 15:53:43,772 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 34 msec 2023-05-05 15:53:43,772 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-05-05 15:53:43,796 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-05-05 15:53:43,801 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-05-05 15:53:43,825 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-05-05 15:53:43,828 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-05-05 15:53:43,830 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-05-05 15:53:43,835 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-05-05 15:53:43,838 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-05-05 15:53:43,841 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-05 15:53:43,843 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-05-05 15:53:43,843 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-05-05 15:53:43,855 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-05-05 15:53:43,860 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-05-05 15:53:43,860 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-05-05 15:53:43,860 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-05 15:53:43,860 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-05-05 15:53:43,860 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-05-05 15:53:43,861 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,44213,1683302021756, sessionid=0x101ef8660cb0000, setting cluster-up flag (Was=false) 2023-05-05 15:53:43,877 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-05 15:53:43,883 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-05-05 15:53:43,885 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,44213,1683302021756 2023-05-05 15:53:43,889 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-05 15:53:43,896 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-05-05 15:53:43,897 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,44213,1683302021756 2023-05-05 15:53:43,900 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.hbase-snapshot/.tmp 2023-05-05 15:53:43,945 INFO [RS:0;jenkins-hbase4:43023] regionserver.HRegionServer(952): ClusterId : b6fc4f2f-8698-4803-99b0-5fcbfd922edc 2023-05-05 15:53:43,945 INFO [RS:2;jenkins-hbase4:40991] regionserver.HRegionServer(952): ClusterId : b6fc4f2f-8698-4803-99b0-5fcbfd922edc 2023-05-05 15:53:43,946 INFO [RS:1;jenkins-hbase4:42799] regionserver.HRegionServer(952): ClusterId : b6fc4f2f-8698-4803-99b0-5fcbfd922edc 2023-05-05 15:53:43,952 DEBUG [RS:1;jenkins-hbase4:42799] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-05-05 15:53:43,952 DEBUG [RS:2;jenkins-hbase4:40991] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-05-05 15:53:43,952 DEBUG [RS:0;jenkins-hbase4:43023] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-05-05 15:53:43,960 DEBUG [RS:1;jenkins-hbase4:42799] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-05-05 15:53:43,960 DEBUG [RS:2;jenkins-hbase4:40991] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-05-05 15:53:43,960 DEBUG [RS:0;jenkins-hbase4:43023] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-05-05 15:53:43,960 DEBUG [RS:2;jenkins-hbase4:40991] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-05-05 15:53:43,960 DEBUG [RS:1;jenkins-hbase4:42799] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-05-05 15:53:43,960 DEBUG [RS:0;jenkins-hbase4:43023] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-05-05 15:53:43,966 DEBUG [RS:1;jenkins-hbase4:42799] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-05-05 15:53:43,966 DEBUG [RS:0;jenkins-hbase4:43023] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-05-05 15:53:43,966 DEBUG [RS:2;jenkins-hbase4:40991] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-05-05 15:53:43,969 DEBUG [RS:1;jenkins-hbase4:42799] zookeeper.ReadOnlyZKClient(139): Connect 0x12dde343 to 127.0.0.1:51118 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-05-05 15:53:43,972 DEBUG [RS:0;jenkins-hbase4:43023] zookeeper.ReadOnlyZKClient(139): Connect 0x16d749ca to 127.0.0.1:51118 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-05-05 15:53:43,972 DEBUG [RS:2;jenkins-hbase4:40991] zookeeper.ReadOnlyZKClient(139): Connect 0x21a06c2b to 127.0.0.1:51118 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-05-05 15:53:44,018 DEBUG [RS:0;jenkins-hbase4:43023] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1810a560, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-05-05 15:53:44,019 DEBUG [RS:2;jenkins-hbase4:40991] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2f67866f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-05-05 15:53:44,019 DEBUG [RS:0;jenkins-hbase4:43023] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1c2f22ba, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-05-05 15:53:44,019 DEBUG [RS:2;jenkins-hbase4:40991] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2b58075b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-05-05 15:53:44,020 DEBUG [RS:1;jenkins-hbase4:42799] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1c1a0f68, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-05-05 15:53:44,020 DEBUG [RS:1;jenkins-hbase4:42799] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@18258833, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-05-05 15:53:44,042 DEBUG [RS:1;jenkins-hbase4:42799] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:42799 2023-05-05 15:53:44,042 DEBUG [RS:2;jenkins-hbase4:40991] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:40991 2023-05-05 15:53:44,046 INFO [RS:2;jenkins-hbase4:40991] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-05-05 15:53:44,046 INFO [RS:1;jenkins-hbase4:42799] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-05-05 15:53:44,047 INFO [RS:2;jenkins-hbase4:40991] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-05-05 15:53:44,047 INFO [RS:1;jenkins-hbase4:42799] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-05-05 15:53:44,047 DEBUG [RS:2;jenkins-hbase4:40991] regionserver.HRegionServer(1023): About to register with Master. 2023-05-05 15:53:44,047 DEBUG [RS:1;jenkins-hbase4:42799] regionserver.HRegionServer(1023): About to register with Master. 2023-05-05 15:53:44,047 DEBUG [RS:0;jenkins-hbase4:43023] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:43023 2023-05-05 15:53:44,048 INFO [RS:0;jenkins-hbase4:43023] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-05-05 15:53:44,048 INFO [RS:0;jenkins-hbase4:43023] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-05-05 15:53:44,048 DEBUG [RS:0;jenkins-hbase4:43023] regionserver.HRegionServer(1023): About to register with Master. 2023-05-05 15:53:44,050 INFO [RS:2;jenkins-hbase4:40991] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,44213,1683302021756 with isa=jenkins-hbase4.apache.org/172.31.14.131:40991, startcode=1683302023016 2023-05-05 15:53:44,050 INFO [RS:0;jenkins-hbase4:43023] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,44213,1683302021756 with isa=jenkins-hbase4.apache.org/172.31.14.131:43023, startcode=1683302022922 2023-05-05 15:53:44,051 INFO [RS:1;jenkins-hbase4:42799] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,44213,1683302021756 with isa=jenkins-hbase4.apache.org/172.31.14.131:42799, startcode=1683302022977 2023-05-05 15:53:44,059 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-05-05 15:53:44,069 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-05-05 15:53:44,069 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-05-05 15:53:44,069 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-05-05 15:53:44,069 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-05-05 15:53:44,069 DEBUG [RS:0;jenkins-hbase4:43023] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-05-05 15:53:44,069 DEBUG [RS:2;jenkins-hbase4:40991] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-05-05 15:53:44,069 DEBUG [RS:1;jenkins-hbase4:42799] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-05-05 15:53:44,069 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-05-05 15:53:44,070 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,070 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-05-05 15:53:44,070 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,071 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302054071 2023-05-05 15:53:44,073 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-05-05 15:53:44,079 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-05-05 15:53:44,079 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-05-05 15:53:44,085 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-05-05 15:53:44,086 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-05-05 15:53:44,095 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-05-05 15:53:44,095 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-05-05 15:53:44,096 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-05-05 15:53:44,096 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-05-05 15:53:44,101 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,104 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-05-05 15:53:44,106 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-05-05 15:53:44,106 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-05-05 15:53:44,115 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-05-05 15:53:44,116 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-05-05 15:53:44,118 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1683302024117,5,FailOnTimeoutGroup] 2023-05-05 15:53:44,119 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1683302024118,5,FailOnTimeoutGroup] 2023-05-05 15:53:44,119 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,119 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-05-05 15:53:44,120 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,121 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,147 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-05-05 15:53:44,147 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54129, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-05-05 15:53:44,147 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53977, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-05-05 15:53:44,147 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43913, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-05-05 15:53:44,148 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-05-05 15:53:44,149 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27 2023-05-05 15:53:44,161 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44213] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:44,162 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44213] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:44,163 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:44,183 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:53:44,185 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-05-05 15:53:44,188 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740/info 2023-05-05 15:53:44,188 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-05-05 15:53:44,190 DEBUG [RS:0;jenkins-hbase4:43023] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27 2023-05-05 15:53:44,190 DEBUG [RS:2;jenkins-hbase4:40991] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27 2023-05-05 15:53:44,190 DEBUG [RS:1;jenkins-hbase4:42799] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27 2023-05-05 15:53:44,190 DEBUG [RS:2;jenkins-hbase4:40991] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:35855 2023-05-05 15:53:44,190 DEBUG [RS:0;jenkins-hbase4:43023] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:35855 2023-05-05 15:53:44,190 DEBUG [RS:2;jenkins-hbase4:40991] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-05-05 15:53:44,190 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:53:44,190 DEBUG [RS:1;jenkins-hbase4:42799] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:35855 2023-05-05 15:53:44,190 DEBUG [RS:0;jenkins-hbase4:43023] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-05-05 15:53:44,191 DEBUG [RS:1;jenkins-hbase4:42799] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-05-05 15:53:44,191 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-05-05 15:53:44,194 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740/rep_barrier 2023-05-05 15:53:44,194 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-05-05 15:53:44,195 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:53:44,196 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-05-05 15:53:44,199 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740/table 2023-05-05 15:53:44,199 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-05 15:53:44,200 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-05-05 15:53:44,200 DEBUG [RS:0;jenkins-hbase4:43023] zookeeper.ZKUtil(162): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:44,200 WARN [RS:0;jenkins-hbase4:43023] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-05-05 15:53:44,201 INFO [RS:0;jenkins-hbase4:43023] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-05-05 15:53:44,201 DEBUG [RS:0;jenkins-hbase4:43023] regionserver.HRegionServer(1947): logDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/WALs/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:44,201 DEBUG [RS:2;jenkins-hbase4:40991] zookeeper.ZKUtil(162): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:44,202 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:53:44,201 DEBUG [RS:1;jenkins-hbase4:42799] zookeeper.ZKUtil(162): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:44,202 WARN [RS:1;jenkins-hbase4:42799] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-05-05 15:53:44,202 WARN [RS:2;jenkins-hbase4:40991] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-05-05 15:53:44,203 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42799,1683302022977] 2023-05-05 15:53:44,203 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,43023,1683302022922] 2023-05-05 15:53:44,203 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,40991,1683302023016] 2023-05-05 15:53:44,204 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740 2023-05-05 15:53:44,204 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740 2023-05-05 15:53:44,203 INFO [RS:1;jenkins-hbase4:42799] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-05-05 15:53:44,203 INFO [RS:2;jenkins-hbase4:40991] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-05-05 15:53:44,206 DEBUG [RS:1;jenkins-hbase4:42799] regionserver.HRegionServer(1947): logDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/WALs/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:44,206 DEBUG [RS:2;jenkins-hbase4:40991] regionserver.HRegionServer(1947): logDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/WALs/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:44,213 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-05-05 15:53:44,216 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-05-05 15:53:44,217 DEBUG [RS:0;jenkins-hbase4:43023] zookeeper.ZKUtil(162): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:44,218 DEBUG [RS:0;jenkins-hbase4:43023] zookeeper.ZKUtil(162): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:44,218 DEBUG [RS:1;jenkins-hbase4:42799] zookeeper.ZKUtil(162): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:44,219 DEBUG [RS:0;jenkins-hbase4:43023] zookeeper.ZKUtil(162): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:44,220 DEBUG [RS:1;jenkins-hbase4:42799] zookeeper.ZKUtil(162): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:44,221 DEBUG [RS:2;jenkins-hbase4:40991] zookeeper.ZKUtil(162): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:44,221 DEBUG [RS:1;jenkins-hbase4:42799] zookeeper.ZKUtil(162): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:44,221 DEBUG [RS:2;jenkins-hbase4:40991] zookeeper.ZKUtil(162): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:44,222 DEBUG [RS:2;jenkins-hbase4:40991] zookeeper.ZKUtil(162): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:44,225 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-05 15:53:44,226 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66735791, jitterRate=-0.005559220910072327}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-05-05 15:53:44,226 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-05-05 15:53:44,226 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-05-05 15:53:44,226 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-05-05 15:53:44,226 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-05-05 15:53:44,226 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-05-05 15:53:44,226 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-05-05 15:53:44,228 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-05-05 15:53:44,228 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-05-05 15:53:44,233 DEBUG [RS:2;jenkins-hbase4:40991] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-05-05 15:53:44,233 DEBUG [RS:0;jenkins-hbase4:43023] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-05-05 15:53:44,233 DEBUG [RS:1;jenkins-hbase4:42799] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-05-05 15:53:44,236 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-05-05 15:53:44,236 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-05-05 15:53:44,247 INFO [RS:1;jenkins-hbase4:42799] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-05-05 15:53:44,248 INFO [RS:0;jenkins-hbase4:43023] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-05-05 15:53:44,247 INFO [RS:2;jenkins-hbase4:40991] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-05-05 15:53:44,250 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-05-05 15:53:44,271 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-05-05 15:53:44,273 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-05-05 15:53:44,275 INFO [RS:0;jenkins-hbase4:43023] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-05-05 15:53:44,275 INFO [RS:2;jenkins-hbase4:40991] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-05-05 15:53:44,276 INFO [RS:1;jenkins-hbase4:42799] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-05-05 15:53:44,280 INFO [RS:0;jenkins-hbase4:43023] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-05-05 15:53:44,280 INFO [RS:2;jenkins-hbase4:40991] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-05-05 15:53:44,280 INFO [RS:0;jenkins-hbase4:43023] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,280 INFO [RS:2;jenkins-hbase4:40991] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,280 INFO [RS:1;jenkins-hbase4:42799] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-05-05 15:53:44,281 INFO [RS:1;jenkins-hbase4:42799] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,281 INFO [RS:0;jenkins-hbase4:43023] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-05-05 15:53:44,281 INFO [RS:2;jenkins-hbase4:40991] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-05-05 15:53:44,282 INFO [RS:1;jenkins-hbase4:42799] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-05-05 15:53:44,290 INFO [RS:2;jenkins-hbase4:40991] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,290 INFO [RS:0;jenkins-hbase4:43023] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,290 INFO [RS:1;jenkins-hbase4:42799] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,290 DEBUG [RS:2;jenkins-hbase4:40991] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,290 DEBUG [RS:0;jenkins-hbase4:43023] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,291 DEBUG [RS:2;jenkins-hbase4:40991] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,291 DEBUG [RS:0;jenkins-hbase4:43023] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,291 DEBUG [RS:2;jenkins-hbase4:40991] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,291 DEBUG [RS:0;jenkins-hbase4:43023] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,291 DEBUG [RS:2;jenkins-hbase4:40991] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,291 DEBUG [RS:0;jenkins-hbase4:43023] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,291 DEBUG [RS:2;jenkins-hbase4:40991] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,291 DEBUG [RS:0;jenkins-hbase4:43023] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,291 DEBUG [RS:2;jenkins-hbase4:40991] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-05-05 15:53:44,291 DEBUG [RS:0;jenkins-hbase4:43023] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-05-05 15:53:44,291 DEBUG [RS:2;jenkins-hbase4:40991] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,291 DEBUG [RS:0;jenkins-hbase4:43023] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,291 DEBUG [RS:2;jenkins-hbase4:40991] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,291 DEBUG [RS:0;jenkins-hbase4:43023] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,291 DEBUG [RS:1;jenkins-hbase4:42799] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,292 DEBUG [RS:0;jenkins-hbase4:43023] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,292 DEBUG [RS:2;jenkins-hbase4:40991] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,292 DEBUG [RS:0;jenkins-hbase4:43023] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,292 DEBUG [RS:2;jenkins-hbase4:40991] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,292 DEBUG [RS:1;jenkins-hbase4:42799] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,292 DEBUG [RS:1;jenkins-hbase4:42799] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,292 DEBUG [RS:1;jenkins-hbase4:42799] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,292 DEBUG [RS:1;jenkins-hbase4:42799] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,292 DEBUG [RS:1;jenkins-hbase4:42799] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-05-05 15:53:44,292 DEBUG [RS:1;jenkins-hbase4:42799] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,292 DEBUG [RS:1;jenkins-hbase4:42799] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,292 DEBUG [RS:1;jenkins-hbase4:42799] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,292 DEBUG [RS:1;jenkins-hbase4:42799] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-05 15:53:44,296 INFO [RS:1;jenkins-hbase4:42799] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,297 INFO [RS:1;jenkins-hbase4:42799] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,297 INFO [RS:1;jenkins-hbase4:42799] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,297 INFO [RS:0;jenkins-hbase4:43023] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,297 INFO [RS:0;jenkins-hbase4:43023] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,297 INFO [RS:0;jenkins-hbase4:43023] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,300 INFO [RS:2;jenkins-hbase4:40991] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,300 INFO [RS:2;jenkins-hbase4:40991] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,300 INFO [RS:2;jenkins-hbase4:40991] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,319 INFO [RS:0;jenkins-hbase4:43023] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-05-05 15:53:44,319 INFO [RS:2;jenkins-hbase4:40991] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-05-05 15:53:44,319 INFO [RS:1;jenkins-hbase4:42799] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-05-05 15:53:44,322 INFO [RS:1;jenkins-hbase4:42799] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42799,1683302022977-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,322 INFO [RS:2;jenkins-hbase4:40991] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40991,1683302023016-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,322 INFO [RS:0;jenkins-hbase4:43023] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43023,1683302022922-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,346 INFO [RS:2;jenkins-hbase4:40991] regionserver.Replication(203): jenkins-hbase4.apache.org,40991,1683302023016 started 2023-05-05 15:53:44,346 INFO [RS:2;jenkins-hbase4:40991] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,40991,1683302023016, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:40991, sessionid=0x101ef8660cb0003 2023-05-05 15:53:44,347 INFO [RS:0;jenkins-hbase4:43023] regionserver.Replication(203): jenkins-hbase4.apache.org,43023,1683302022922 started 2023-05-05 15:53:44,347 DEBUG [RS:2;jenkins-hbase4:40991] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-05-05 15:53:44,347 INFO [RS:0;jenkins-hbase4:43023] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,43023,1683302022922, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:43023, sessionid=0x101ef8660cb0001 2023-05-05 15:53:44,347 DEBUG [RS:2;jenkins-hbase4:40991] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:44,347 DEBUG [RS:0;jenkins-hbase4:43023] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-05-05 15:53:44,348 DEBUG [RS:2;jenkins-hbase4:40991] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40991,1683302023016' 2023-05-05 15:53:44,348 DEBUG [RS:0;jenkins-hbase4:43023] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:44,348 DEBUG [RS:2;jenkins-hbase4:40991] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:44,348 DEBUG [RS:0;jenkins-hbase4:43023] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43023,1683302022922' 2023-05-05 15:53:44,349 DEBUG [RS:0;jenkins-hbase4:43023] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:44,350 DEBUG [RS:2;jenkins-hbase4:40991] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:44,350 DEBUG [RS:0;jenkins-hbase4:43023] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:44,350 DEBUG [RS:0;jenkins-hbase4:43023] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-05-05 15:53:44,350 DEBUG [RS:2;jenkins-hbase4:40991] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-05-05 15:53:44,350 DEBUG [RS:0;jenkins-hbase4:43023] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-05-05 15:53:44,351 DEBUG [RS:2;jenkins-hbase4:40991] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-05-05 15:53:44,351 DEBUG [RS:0;jenkins-hbase4:43023] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:44,351 DEBUG [RS:2;jenkins-hbase4:40991] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:44,351 DEBUG [RS:0;jenkins-hbase4:43023] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43023,1683302022922' 2023-05-05 15:53:44,351 DEBUG [RS:0;jenkins-hbase4:43023] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-05-05 15:53:44,351 DEBUG [RS:2;jenkins-hbase4:40991] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40991,1683302023016' 2023-05-05 15:53:44,351 DEBUG [RS:2;jenkins-hbase4:40991] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-05-05 15:53:44,352 DEBUG [RS:2;jenkins-hbase4:40991] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-05-05 15:53:44,352 DEBUG [RS:0;jenkins-hbase4:43023] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-05-05 15:53:44,352 INFO [RS:1;jenkins-hbase4:42799] regionserver.Replication(203): jenkins-hbase4.apache.org,42799,1683302022977 started 2023-05-05 15:53:44,352 DEBUG [RS:2;jenkins-hbase4:40991] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-05-05 15:53:44,352 INFO [RS:1;jenkins-hbase4:42799] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,42799,1683302022977, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42799, sessionid=0x101ef8660cb0002 2023-05-05 15:53:44,353 DEBUG [RS:0;jenkins-hbase4:43023] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-05-05 15:53:44,353 INFO [RS:2;jenkins-hbase4:40991] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-05-05 15:53:44,353 INFO [RS:0;jenkins-hbase4:43023] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-05-05 15:53:44,353 DEBUG [RS:1;jenkins-hbase4:42799] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-05-05 15:53:44,353 INFO [RS:0;jenkins-hbase4:43023] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-05-05 15:53:44,353 INFO [RS:2;jenkins-hbase4:40991] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-05-05 15:53:44,354 DEBUG [RS:1;jenkins-hbase4:42799] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:44,354 DEBUG [RS:1;jenkins-hbase4:42799] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42799,1683302022977' 2023-05-05 15:53:44,354 DEBUG [RS:1;jenkins-hbase4:42799] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:44,355 DEBUG [RS:1;jenkins-hbase4:42799] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:44,356 DEBUG [RS:1;jenkins-hbase4:42799] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-05-05 15:53:44,356 DEBUG [RS:1;jenkins-hbase4:42799] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-05-05 15:53:44,356 DEBUG [RS:1;jenkins-hbase4:42799] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:44,356 DEBUG [RS:1;jenkins-hbase4:42799] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42799,1683302022977' 2023-05-05 15:53:44,356 DEBUG [RS:1;jenkins-hbase4:42799] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-05-05 15:53:44,356 DEBUG [RS:1;jenkins-hbase4:42799] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-05-05 15:53:44,357 DEBUG [RS:1;jenkins-hbase4:42799] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-05-05 15:53:44,357 INFO [RS:1;jenkins-hbase4:42799] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-05-05 15:53:44,357 INFO [RS:1;jenkins-hbase4:42799] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-05-05 15:53:44,425 DEBUG [jenkins-hbase4:44213] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-05-05 15:53:44,429 DEBUG [jenkins-hbase4:44213] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-05 15:53:44,435 DEBUG [jenkins-hbase4:44213] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-05 15:53:44,435 DEBUG [jenkins-hbase4:44213] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-05 15:53:44,435 DEBUG [jenkins-hbase4:44213] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-05 15:53:44,435 DEBUG [jenkins-hbase4:44213] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-05 15:53:44,439 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,40991,1683302023016, state=OPENING 2023-05-05 15:53:44,447 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-05-05 15:53:44,448 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-05 15:53:44,449 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-05-05 15:53:44,452 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:53:44,465 INFO [RS:0;jenkins-hbase4:43023] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43023%2C1683302022922, suffix=, logDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/WALs/jenkins-hbase4.apache.org,43023,1683302022922, archiveDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/oldWALs, maxLogs=32 2023-05-05 15:53:44,465 INFO [RS:2;jenkins-hbase4:40991] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40991%2C1683302023016, suffix=, logDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/WALs/jenkins-hbase4.apache.org,40991,1683302023016, archiveDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/oldWALs, maxLogs=32 2023-05-05 15:53:44,467 INFO [RS:1;jenkins-hbase4:42799] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42799%2C1683302022977, suffix=, logDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/WALs/jenkins-hbase4.apache.org,42799,1683302022977, archiveDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/oldWALs, maxLogs=32 2023-05-05 15:53:44,505 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36035,DS-3097abad-0a2f-49cd-8a8a-8998355904cb,DISK] 2023-05-05 15:53:44,508 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40251,DS-fffa5dcb-dda3-4efc-84b4-1fd59220aa20,DISK] 2023-05-05 15:53:44,508 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35557,DS-5f1bb721-5edc-405a-aaad-54574a665d94,DISK] 2023-05-05 15:53:44,509 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36035,DS-3097abad-0a2f-49cd-8a8a-8998355904cb,DISK] 2023-05-05 15:53:44,510 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40251,DS-fffa5dcb-dda3-4efc-84b4-1fd59220aa20,DISK] 2023-05-05 15:53:44,510 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35557,DS-5f1bb721-5edc-405a-aaad-54574a665d94,DISK] 2023-05-05 15:53:44,517 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36035,DS-3097abad-0a2f-49cd-8a8a-8998355904cb,DISK] 2023-05-05 15:53:44,517 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35557,DS-5f1bb721-5edc-405a-aaad-54574a665d94,DISK] 2023-05-05 15:53:44,518 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40251,DS-fffa5dcb-dda3-4efc-84b4-1fd59220aa20,DISK] 2023-05-05 15:53:44,527 INFO [RS:2;jenkins-hbase4:40991] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/WALs/jenkins-hbase4.apache.org,40991,1683302023016/jenkins-hbase4.apache.org%2C40991%2C1683302023016.1683302024471 2023-05-05 15:53:44,527 INFO [RS:0;jenkins-hbase4:43023] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/WALs/jenkins-hbase4.apache.org,43023,1683302022922/jenkins-hbase4.apache.org%2C43023%2C1683302022922.1683302024471 2023-05-05 15:53:44,532 DEBUG [RS:2;jenkins-hbase4:40991] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36035,DS-3097abad-0a2f-49cd-8a8a-8998355904cb,DISK], DatanodeInfoWithStorage[127.0.0.1:35557,DS-5f1bb721-5edc-405a-aaad-54574a665d94,DISK], DatanodeInfoWithStorage[127.0.0.1:40251,DS-fffa5dcb-dda3-4efc-84b4-1fd59220aa20,DISK]] 2023-05-05 15:53:44,532 DEBUG [RS:0;jenkins-hbase4:43023] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36035,DS-3097abad-0a2f-49cd-8a8a-8998355904cb,DISK], DatanodeInfoWithStorage[127.0.0.1:40251,DS-fffa5dcb-dda3-4efc-84b4-1fd59220aa20,DISK], DatanodeInfoWithStorage[127.0.0.1:35557,DS-5f1bb721-5edc-405a-aaad-54574a665d94,DISK]] 2023-05-05 15:53:44,536 INFO [RS:1;jenkins-hbase4:42799] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/WALs/jenkins-hbase4.apache.org,42799,1683302022977/jenkins-hbase4.apache.org%2C42799%2C1683302022977.1683302024471 2023-05-05 15:53:44,537 DEBUG [RS:1;jenkins-hbase4:42799] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35557,DS-5f1bb721-5edc-405a-aaad-54574a665d94,DISK], DatanodeInfoWithStorage[127.0.0.1:36035,DS-3097abad-0a2f-49cd-8a8a-8998355904cb,DISK], DatanodeInfoWithStorage[127.0.0.1:40251,DS-fffa5dcb-dda3-4efc-84b4-1fd59220aa20,DISK]] 2023-05-05 15:53:44,635 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:44,637 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-05-05 15:53:44,640 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60928, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-05-05 15:53:44,653 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-05-05 15:53:44,654 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-05-05 15:53:44,657 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40991%2C1683302023016.meta, suffix=.meta, logDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/WALs/jenkins-hbase4.apache.org,40991,1683302023016, archiveDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/oldWALs, maxLogs=32 2023-05-05 15:53:44,680 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40251,DS-fffa5dcb-dda3-4efc-84b4-1fd59220aa20,DISK] 2023-05-05 15:53:44,680 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36035,DS-3097abad-0a2f-49cd-8a8a-8998355904cb,DISK] 2023-05-05 15:53:44,682 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35557,DS-5f1bb721-5edc-405a-aaad-54574a665d94,DISK] 2023-05-05 15:53:44,686 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/WALs/jenkins-hbase4.apache.org,40991,1683302023016/jenkins-hbase4.apache.org%2C40991%2C1683302023016.meta.1683302024658.meta 2023-05-05 15:53:44,687 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40251,DS-fffa5dcb-dda3-4efc-84b4-1fd59220aa20,DISK], DatanodeInfoWithStorage[127.0.0.1:36035,DS-3097abad-0a2f-49cd-8a8a-8998355904cb,DISK], DatanodeInfoWithStorage[127.0.0.1:35557,DS-5f1bb721-5edc-405a-aaad-54574a665d94,DISK]] 2023-05-05 15:53:44,687 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-05-05 15:53:44,691 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:53:44,692 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-05-05 15:53:44,710 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-05-05 15:53:44,713 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-05-05 15:53:44,717 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-05-05 15:53:44,718 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:53:44,718 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-05-05 15:53:44,718 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-05-05 15:53:44,721 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-05-05 15:53:44,724 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740/info 2023-05-05 15:53:44,724 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740/info 2023-05-05 15:53:44,724 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-05-05 15:53:44,725 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:53:44,725 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-05-05 15:53:44,727 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740/rep_barrier 2023-05-05 15:53:44,727 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740/rep_barrier 2023-05-05 15:53:44,727 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-05-05 15:53:44,728 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:53:44,728 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-05-05 15:53:44,729 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740/table 2023-05-05 15:53:44,730 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740/table 2023-05-05 15:53:44,730 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-05-05 15:53:44,731 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:53:44,732 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740 2023-05-05 15:53:44,735 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740 2023-05-05 15:53:44,738 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-05-05 15:53:44,740 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-05-05 15:53:44,742 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75324668, jitterRate=0.12242501974105835}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-05-05 15:53:44,742 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-05-05 15:53:44,751 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1683302024627 2023-05-05 15:53:44,771 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-05-05 15:53:44,772 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-05-05 15:53:44,773 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,40991,1683302023016, state=OPEN 2023-05-05 15:53:44,775 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-05-05 15:53:44,775 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-05-05 15:53:44,779 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-05-05 15:53:44,779 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,40991,1683302023016 in 323 msec 2023-05-05 15:53:44,785 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-05-05 15:53:44,785 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 532 msec 2023-05-05 15:53:44,790 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 846 msec 2023-05-05 15:53:44,791 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1683302024790, completionTime=-1 2023-05-05 15:53:44,791 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-05-05 15:53:44,791 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:53:44,854 DEBUG [hconnection-0x603c4d85-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-05-05 15:53:44,858 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60944, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-05-05 15:53:44,876 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:53:44,876 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302084876 2023-05-05 15:53:44,876 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302144876 2023-05-05 15:53:44,876 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 85 msec 2023-05-05 15:53:44,903 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44213,1683302021756-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,903 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44213,1683302021756-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,904 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44213,1683302021756-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,906 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:44213, period=300000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,907 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-05-05 15:53:44,918 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-05-05 15:53:44,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-05-05 15:53:44,927 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-05-05 15:53:44,936 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-05-05 15:53:44,938 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-05-05 15:53:44,941 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-05 15:53:44,965 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/hbase/namespace/d05f6ba26fbb949a8331f3129bf8461a 2023-05-05 15:53:44,967 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/hbase/namespace/d05f6ba26fbb949a8331f3129bf8461a empty. 2023-05-05 15:53:44,968 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/hbase/namespace/d05f6ba26fbb949a8331f3129bf8461a 2023-05-05 15:53:44,968 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-05-05 15:53:45,010 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-05-05 15:53:45,012 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => d05f6ba26fbb949a8331f3129bf8461a, NAME => 'hbase:namespace,,1683302024926.d05f6ba26fbb949a8331f3129bf8461a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp 2023-05-05 15:53:45,028 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1683302024926.d05f6ba26fbb949a8331f3129bf8461a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:53:45,028 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing d05f6ba26fbb949a8331f3129bf8461a, disabling compactions & flushes 2023-05-05 15:53:45,029 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1683302024926.d05f6ba26fbb949a8331f3129bf8461a. 2023-05-05 15:53:45,029 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1683302024926.d05f6ba26fbb949a8331f3129bf8461a. 2023-05-05 15:53:45,029 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1683302024926.d05f6ba26fbb949a8331f3129bf8461a. after waiting 0 ms 2023-05-05 15:53:45,029 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1683302024926.d05f6ba26fbb949a8331f3129bf8461a. 2023-05-05 15:53:45,029 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1683302024926.d05f6ba26fbb949a8331f3129bf8461a. 2023-05-05 15:53:45,029 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for d05f6ba26fbb949a8331f3129bf8461a: 2023-05-05 15:53:45,033 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-05-05 15:53:45,048 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1683302024926.d05f6ba26fbb949a8331f3129bf8461a.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1683302025036"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302025036"}]},"ts":"1683302025036"} 2023-05-05 15:53:45,075 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-05 15:53:45,077 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-05 15:53:45,082 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302025077"}]},"ts":"1683302025077"} 2023-05-05 15:53:45,086 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-05-05 15:53:45,091 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-05 15:53:45,092 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-05 15:53:45,092 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-05 15:53:45,092 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-05 15:53:45,092 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-05 15:53:45,094 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=d05f6ba26fbb949a8331f3129bf8461a, ASSIGN}] 2023-05-05 15:53:45,099 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=d05f6ba26fbb949a8331f3129bf8461a, ASSIGN 2023-05-05 15:53:45,101 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=d05f6ba26fbb949a8331f3129bf8461a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43023,1683302022922; forceNewPlan=false, retain=false 2023-05-05 15:53:45,253 INFO [jenkins-hbase4:44213] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-05 15:53:45,255 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:45,255 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1683302024926.d05f6ba26fbb949a8331f3129bf8461a.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1683302025254"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302025254"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302025254"}]},"ts":"1683302025254"} 2023-05-05 15:53:45,260 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure d05f6ba26fbb949a8331f3129bf8461a, server=jenkins-hbase4.apache.org,43023,1683302022922}] 2023-05-05 15:53:45,415 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:45,415 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-05-05 15:53:45,419 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47046, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-05-05 15:53:45,425 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1683302024926.d05f6ba26fbb949a8331f3129bf8461a. 2023-05-05 15:53:45,427 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d05f6ba26fbb949a8331f3129bf8461a, NAME => 'hbase:namespace,,1683302024926.d05f6ba26fbb949a8331f3129bf8461a.', STARTKEY => '', ENDKEY => ''} 2023-05-05 15:53:45,428 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:53:45,429 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace d05f6ba26fbb949a8331f3129bf8461a 2023-05-05 15:53:45,429 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1683302024926.d05f6ba26fbb949a8331f3129bf8461a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:53:45,429 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d05f6ba26fbb949a8331f3129bf8461a 2023-05-05 15:53:45,429 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d05f6ba26fbb949a8331f3129bf8461a 2023-05-05 15:53:45,431 INFO [StoreOpener-d05f6ba26fbb949a8331f3129bf8461a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region d05f6ba26fbb949a8331f3129bf8461a 2023-05-05 15:53:45,434 DEBUG [StoreOpener-d05f6ba26fbb949a8331f3129bf8461a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/namespace/d05f6ba26fbb949a8331f3129bf8461a/info 2023-05-05 15:53:45,434 DEBUG [StoreOpener-d05f6ba26fbb949a8331f3129bf8461a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/namespace/d05f6ba26fbb949a8331f3129bf8461a/info 2023-05-05 15:53:45,434 INFO [StoreOpener-d05f6ba26fbb949a8331f3129bf8461a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d05f6ba26fbb949a8331f3129bf8461a columnFamilyName info 2023-05-05 15:53:45,435 INFO [StoreOpener-d05f6ba26fbb949a8331f3129bf8461a-1] regionserver.HStore(310): Store=d05f6ba26fbb949a8331f3129bf8461a/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:53:45,437 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/namespace/d05f6ba26fbb949a8331f3129bf8461a 2023-05-05 15:53:45,438 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/namespace/d05f6ba26fbb949a8331f3129bf8461a 2023-05-05 15:53:45,442 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d05f6ba26fbb949a8331f3129bf8461a 2023-05-05 15:53:45,446 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/namespace/d05f6ba26fbb949a8331f3129bf8461a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-05 15:53:45,447 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d05f6ba26fbb949a8331f3129bf8461a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63645960, jitterRate=-0.051601290702819824}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-05-05 15:53:45,447 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d05f6ba26fbb949a8331f3129bf8461a: 2023-05-05 15:53:45,450 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1683302024926.d05f6ba26fbb949a8331f3129bf8461a., pid=6, masterSystemTime=1683302025415 2023-05-05 15:53:45,454 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1683302024926.d05f6ba26fbb949a8331f3129bf8461a. 2023-05-05 15:53:45,455 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1683302024926.d05f6ba26fbb949a8331f3129bf8461a. 2023-05-05 15:53:45,456 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:45,456 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1683302024926.d05f6ba26fbb949a8331f3129bf8461a.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1683302025455"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302025455"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302025455"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302025455"}]},"ts":"1683302025455"} 2023-05-05 15:53:45,465 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-05-05 15:53:45,465 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure d05f6ba26fbb949a8331f3129bf8461a, server=jenkins-hbase4.apache.org,43023,1683302022922 in 201 msec 2023-05-05 15:53:45,469 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-05-05 15:53:45,469 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=d05f6ba26fbb949a8331f3129bf8461a, ASSIGN in 371 msec 2023-05-05 15:53:45,470 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-05 15:53:45,471 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302025470"}]},"ts":"1683302025470"} 2023-05-05 15:53:45,473 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-05-05 15:53:45,477 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-05-05 15:53:45,480 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 550 msec 2023-05-05 15:53:45,538 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-05-05 15:53:45,541 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-05-05 15:53:45,541 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-05 15:53:45,562 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-05-05 15:53:45,565 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47052, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-05-05 15:53:45,577 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-05-05 15:53:45,595 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-05-05 15:53:45,601 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 31 msec 2023-05-05 15:53:45,611 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:53:45,624 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-05-05 15:53:45,631 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 19 msec 2023-05-05 15:53:45,637 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-05-05 15:53:45,640 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-05-05 15:53:45,641 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.558sec 2023-05-05 15:53:45,643 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-05-05 15:53:45,645 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-05-05 15:53:45,645 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-05-05 15:53:45,646 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44213,1683302021756-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-05-05 15:53:45,647 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44213,1683302021756-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-05-05 15:53:45,653 DEBUG [Listener at localhost/35117] zookeeper.ReadOnlyZKClient(139): Connect 0x745e3448 to 127.0.0.1:51118 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-05-05 15:53:45,658 DEBUG [Listener at localhost/35117] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5c62b4e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-05-05 15:53:45,658 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-05-05 15:53:45,672 DEBUG [hconnection-0x6277894c-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-05-05 15:53:45,687 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60952, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-05-05 15:53:45,697 INFO [Listener at localhost/35117] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,44213,1683302021756 2023-05-05 15:53:45,728 INFO [Listener at localhost/35117] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=359, OpenFileDescriptor=695, MaxFileDescriptor=60000, SystemLoadAverage=369, ProcessCount=179, AvailableMemoryMB=5396 2023-05-05 15:53:45,732 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-05 15:53:45,741 DEBUG [Listener at localhost/35117] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-05-05 15:53:45,744 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58500, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-05-05 15:53:45,756 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-05-05 15:53:45,756 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-05 15:53:45,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-05 15:53:45,762 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:45,764 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-05-05 15:53:45,766 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-05 15:53:45,769 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:45,770 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1 empty. 2023-05-05 15:53:45,772 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:45,772 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-05-05 15:53:45,794 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-05-05 15:53:45,796 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => 2faa234e4e385b82c11337930631d2a1, NAME => 'testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp 2023-05-05 15:53:45,817 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:53:45,817 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1604): Closing 2faa234e4e385b82c11337930631d2a1, disabling compactions & flushes 2023-05-05 15:53:45,817 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. 2023-05-05 15:53:45,817 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. 2023-05-05 15:53:45,817 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. after waiting 0 ms 2023-05-05 15:53:45,817 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. 2023-05-05 15:53:45,817 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. 2023-05-05 15:53:45,817 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1558): Region close journal for 2faa234e4e385b82c11337930631d2a1: 2023-05-05 15:53:45,822 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-05-05 15:53:45,823 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1683302025823"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302025823"}]},"ts":"1683302025823"} 2023-05-05 15:53:45,826 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-05 15:53:45,827 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-05 15:53:45,828 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302025827"}]},"ts":"1683302025827"} 2023-05-05 15:53:45,830 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-05-05 15:53:45,835 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-05 15:53:45,836 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-05 15:53:45,836 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-05 15:53:45,836 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-05 15:53:45,836 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-05 15:53:45,836 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=2faa234e4e385b82c11337930631d2a1, ASSIGN}] 2023-05-05 15:53:45,839 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=2faa234e4e385b82c11337930631d2a1, ASSIGN 2023-05-05 15:53:45,840 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=2faa234e4e385b82c11337930631d2a1, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42799,1683302022977; forceNewPlan=false, retain=false 2023-05-05 15:53:45,991 INFO [jenkins-hbase4:44213] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-05 15:53:45,992 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=2faa234e4e385b82c11337930631d2a1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:45,992 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1683302025992"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302025992"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302025992"}]},"ts":"1683302025992"} 2023-05-05 15:53:45,996 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 2faa234e4e385b82c11337930631d2a1, server=jenkins-hbase4.apache.org,42799,1683302022977}] 2023-05-05 15:53:46,150 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,151 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-05-05 15:53:46,154 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45654, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-05-05 15:53:46,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. 2023-05-05 15:53:46,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2faa234e4e385b82c11337930631d2a1, NAME => 'testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1.', STARTKEY => '', ENDKEY => ''} 2023-05-05 15:53:46,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:53:46,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:46,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:53:46,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:46,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:46,163 INFO [StoreOpener-2faa234e4e385b82c11337930631d2a1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:46,166 DEBUG [StoreOpener-2faa234e4e385b82c11337930631d2a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1 2023-05-05 15:53:46,166 DEBUG [StoreOpener-2faa234e4e385b82c11337930631d2a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1 2023-05-05 15:53:46,167 INFO [StoreOpener-2faa234e4e385b82c11337930631d2a1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2faa234e4e385b82c11337930631d2a1 columnFamilyName cf1 2023-05-05 15:53:46,168 INFO [StoreOpener-2faa234e4e385b82c11337930631d2a1-1] regionserver.HStore(310): Store=2faa234e4e385b82c11337930631d2a1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:53:46,168 INFO [StoreOpener-2faa234e4e385b82c11337930631d2a1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:46,170 DEBUG [StoreOpener-2faa234e4e385b82c11337930631d2a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2 2023-05-05 15:53:46,171 DEBUG [StoreOpener-2faa234e4e385b82c11337930631d2a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2 2023-05-05 15:53:46,171 INFO [StoreOpener-2faa234e4e385b82c11337930631d2a1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2faa234e4e385b82c11337930631d2a1 columnFamilyName cf2 2023-05-05 15:53:46,172 INFO [StoreOpener-2faa234e4e385b82c11337930631d2a1-1] regionserver.HStore(310): Store=2faa234e4e385b82c11337930631d2a1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:53:46,174 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:46,175 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:46,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-05 15:53:46,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:46,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-05 15:53:46,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2faa234e4e385b82c11337930631d2a1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68491514, jitterRate=0.020603090524673462}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-05 15:53:46,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2faa234e4e385b82c11337930631d2a1: 2023-05-05 15:53:46,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1., pid=11, masterSystemTime=1683302026150 2023-05-05 15:53:46,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. 2023-05-05 15:53:46,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. 2023-05-05 15:53:46,196 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=2faa234e4e385b82c11337930631d2a1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,197 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1683302026196"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302026196"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302026196"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302026196"}]},"ts":"1683302026196"} 2023-05-05 15:53:46,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-05-05 15:53:46,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 2faa234e4e385b82c11337930631d2a1, server=jenkins-hbase4.apache.org,42799,1683302022977 in 204 msec 2023-05-05 15:53:46,208 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-05-05 15:53:46,208 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=2faa234e4e385b82c11337930631d2a1, ASSIGN in 368 msec 2023-05-05 15:53:46,209 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-05 15:53:46,209 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302026209"}]},"ts":"1683302026209"} 2023-05-05 15:53:46,212 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-05-05 15:53:46,215 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-05-05 15:53:46,218 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 455 msec 2023-05-05 15:53:46,268 DEBUG [Listener at localhost/35117] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-05-05 15:53:46,273 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45666, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-05-05 15:53:46,295 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:53:46,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,303 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-05-05 15:53:46,303 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:46,304 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-05-05 15:53:46,304 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-05-05 15:53:46,305 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,305 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,306 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:46,306 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:46,306 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:46,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:46,306 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:46,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:46,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:46,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:46,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:46,307 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,307 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:53:46,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:46,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:46,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,308 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:46,308 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,309 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,310 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,310 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,310 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,311 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-05 15:53:46,311 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-05 15:53:46,311 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:46,311 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:46,311 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-05 15:53:46,312 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-05 15:53:46,311 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-05 15:53:46,312 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-05 15:53:46,312 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:46,312 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-05 15:53:46,313 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-05 15:53:46,313 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-05 15:53:46,313 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-05 15:53:46,313 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:53:46,313 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:53:46,313 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. 2023-05-05 15:53:46,313 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. started... 2023-05-05 15:53:46,314 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2faa234e4e385b82c11337930631d2a1 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-05 15:53:46,315 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,315 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,315 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,315 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:46,315 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,316 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:46,387 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf1/d69a911d9c294c03b7032ffbd281155f 2023-05-05 15:53:46,446 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf2/33cfd2a14784445ba210b3c709d236ec 2023-05-05 15:53:46,455 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf1/d69a911d9c294c03b7032ffbd281155f as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/d69a911d9c294c03b7032ffbd281155f 2023-05-05 15:53:46,472 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/d69a911d9c294c03b7032ffbd281155f, entries=2, sequenceid=6, filesize=4.8 K 2023-05-05 15:53:46,474 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf2/33cfd2a14784445ba210b3c709d236ec as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/33cfd2a14784445ba210b3c709d236ec 2023-05-05 15:53:46,483 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/33cfd2a14784445ba210b3c709d236ec, entries=2, sequenceid=6, filesize=4.8 K 2023-05-05 15:53:46,485 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 2faa234e4e385b82c11337930631d2a1 in 171ms, sequenceid=6, compaction requested=false 2023-05-05 15:53:46,487 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-05-05 15:53:46,490 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2faa234e4e385b82c11337930631d2a1: 2023-05-05 15:53:46,490 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. 2023-05-05 15:53:46,490 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:53:46,491 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:53:46,491 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,491 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-05 15:53:46,491 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-05 15:53:46,494 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,494 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:46,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:46,495 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,495 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:46,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:46,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:46,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:46,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:46,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:46,498 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-05-05 15:53:46,498 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-05-05 15:53:46,498 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@12021de8[Count = 0] remaining members to acquire global barrier 2023-05-05 15:53:46,498 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,507 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,507 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,507 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,508 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-05 15:53:46,508 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,508 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-05 15:53:46,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,508 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-05 15:53:46,508 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:53:46,508 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:53:46,508 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-05 15:53:46,509 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:53:46,508 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-05 15:53:46,509 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-05 15:53:46,509 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:53:46,512 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-05 15:53:46,512 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:46,512 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-05 15:53:46,514 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-05 15:53:46,514 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:46,514 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-05 15:53:46,514 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-05 15:53:46,514 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,514 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:46,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:46,515 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-05 15:53:46,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:46,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:46,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:46,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:46,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:46,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:46,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:46,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:46,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,42799,1683302022977': 2023-05-05 15:53:46,520 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-05-05 15:53:46,520 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-05-05 15:53:46,520 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:53:46,520 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:53:46,520 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,520 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:53:46,522 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,522 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,523 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,523 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:46,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,523 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:46,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,522 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:46,522 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:46,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:46,523 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:46,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:46,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:46,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:46,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:46,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:46,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:46,524 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:46,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,524 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:46,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:46,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,527 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:46,527 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:46,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:46,528 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:46,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:46,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:46,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:46,542 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:46,542 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:46,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:46,542 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:46,542 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,542 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:46,542 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:46,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:46,543 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:46,542 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:46,543 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter because node does not exist (not an error) 2023-05-05 15:53:46,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:46,543 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-05-05 15:53:46,543 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,543 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:46,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:46,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:46,543 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:46,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:46,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:46,543 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,543 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:53:46,544 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:46,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:46,545 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,545 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:46,545 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,545 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,545 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:46,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:46,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:46,547 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-05-05 15:53:46,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:46,547 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:53:46,647 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:53:46,652 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:53:46,679 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:53:46,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-05-05 15:53:46,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,682 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-05-05 15:53:46,682 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:46,683 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-05-05 15:53:46,683 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-05-05 15:53:46,683 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,683 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,686 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,686 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:46,686 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:46,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:46,686 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:46,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:46,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:46,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:46,686 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,686 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:53:46,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:46,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:46,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,687 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,687 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:46,688 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,688 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:46,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,688 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-05 15:53:46,688 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:46,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,688 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,688 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-05 15:53:46,689 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,688 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,689 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-05 15:53:46,689 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-05 15:53:46,689 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:46,689 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:46,690 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-05 15:53:46,690 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-05 15:53:46,690 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-05 15:53:46,690 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,690 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-05 15:53:46,690 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-05 15:53:46,690 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:46,690 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-05 15:53:46,693 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:53:46,693 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-05 15:53:46,693 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. 2023-05-05 15:53:46,693 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-05 15:53:46,693 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-05 15:53:46,693 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:53:46,693 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. started... 2023-05-05 15:53:46,694 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2faa234e4e385b82c11337930631d2a1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:53:46,695 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,696 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,696 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,696 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:46,696 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,696 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:46,719 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf1/043c3df5b6d24e3cbf5e78974ea62314 2023-05-05 15:53:46,748 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf2/4991bfadfb32442e904704119d917f44 2023-05-05 15:53:46,757 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf1/043c3df5b6d24e3cbf5e78974ea62314 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/043c3df5b6d24e3cbf5e78974ea62314 2023-05-05 15:53:46,766 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/043c3df5b6d24e3cbf5e78974ea62314, entries=10, sequenceid=19, filesize=5.0 K 2023-05-05 15:53:46,768 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf2/4991bfadfb32442e904704119d917f44 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/4991bfadfb32442e904704119d917f44 2023-05-05 15:53:46,778 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/4991bfadfb32442e904704119d917f44, entries=10, sequenceid=19, filesize=5.0 K 2023-05-05 15:53:46,779 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2faa234e4e385b82c11337930631d2a1 in 85ms, sequenceid=19, compaction requested=false 2023-05-05 15:53:46,779 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2faa234e4e385b82c11337930631d2a1: 2023-05-05 15:53:46,779 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. 2023-05-05 15:53:46,779 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:53:46,780 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:53:46,780 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,780 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-05 15:53:46,780 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-05 15:53:46,782 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,782 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:46,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:46,783 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,783 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:46,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:46,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:46,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:46,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:46,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:46,786 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-05-05 15:53:46,786 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6aa05aa8[Count = 0] remaining members to acquire global barrier 2023-05-05 15:53:46,786 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-05-05 15:53:46,786 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,788 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,789 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,788 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,789 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-05 15:53:46,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,789 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,789 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-05 15:53:46,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,789 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:53:46,789 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:53:46,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,789 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-05 15:53:46,789 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-05 15:53:46,789 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-05 15:53:46,789 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-05 15:53:46,790 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:53:46,789 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:53:46,792 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-05 15:53:46,792 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:46,792 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-05 15:53:46,793 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,793 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-05 15:53:46,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:46,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:46,793 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:46,793 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-05 15:53:46,794 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-05 15:53:46,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:46,794 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:46,794 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-05 15:53:46,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:46,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:46,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:46,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:46,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:46,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:46,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,42799,1683302022977': 2023-05-05 15:53:46,799 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-05-05 15:53:46,799 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-05-05 15:53:46,799 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:53:46,799 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:53:46,799 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,799 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:53:46,802 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,802 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,802 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,802 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,802 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:46,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:46,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:46,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,802 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:46,802 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:46,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:46,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:46,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:46,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:46,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:46,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:46,804 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:46,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,805 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:46,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:46,805 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:46,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:46,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:46,808 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:46,808 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:46,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,809 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:46,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:46,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:46,819 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:46,819 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:46,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:46,819 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:46,819 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:46,819 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-05-05 15:53:46,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:46,819 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:46,819 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,819 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:46,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:46,819 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:46,819 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:46,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:46,820 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:46,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:46,820 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:53:46,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:46,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:46,821 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:46,821 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,821 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:46,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:46,821 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-05-05 15:53:46,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:46,821 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,821 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:53:46,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:46,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:46,821 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:46,822 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,822 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,822 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,922 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:53:46,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:53:46,947 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:53:46,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-05-05 15:53:46,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,950 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-05-05 15:53:46,950 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:46,951 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-05-05 15:53:46,951 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-05-05 15:53:46,951 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,951 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,953 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:46,953 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:46,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:46,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:46,953 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,953 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:46,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:46,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:46,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:46,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:46,953 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:46,953 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:53:46,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,954 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,954 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:46,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,954 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,954 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:46,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,955 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:46,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,955 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-05 15:53:46,957 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-05 15:53:46,957 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,955 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,957 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-05 15:53:46,957 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,957 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:46,957 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-05 15:53:46,958 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-05 15:53:46,957 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-05 15:53:46,958 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:46,958 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,959 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-05 15:53:46,959 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-05 15:53:46,959 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-05 15:53:46,959 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. 2023-05-05 15:53:46,958 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:53:46,959 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. started... 2023-05-05 15:53:46,959 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-05 15:53:46,959 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:46,959 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:53:46,960 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2faa234e4e385b82c11337930631d2a1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:53:46,960 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-05 15:53:46,960 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-05 15:53:46,960 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-05 15:53:46,962 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,963 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,963 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,963 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:46,964 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:46,964 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:46,979 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf1/17846580efc44fc9b1bbdfb8e6f5a0de 2023-05-05 15:53:47,008 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf2/1903627febdd4806832679268b98ef95 2023-05-05 15:53:47,017 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf1/17846580efc44fc9b1bbdfb8e6f5a0de as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/17846580efc44fc9b1bbdfb8e6f5a0de 2023-05-05 15:53:47,025 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/17846580efc44fc9b1bbdfb8e6f5a0de, entries=10, sequenceid=32, filesize=5.0 K 2023-05-05 15:53:47,026 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf2/1903627febdd4806832679268b98ef95 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/1903627febdd4806832679268b98ef95 2023-05-05 15:53:47,034 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/1903627febdd4806832679268b98ef95, entries=10, sequenceid=32, filesize=5.0 K 2023-05-05 15:53:47,035 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2faa234e4e385b82c11337930631d2a1 in 76ms, sequenceid=32, compaction requested=true 2023-05-05 15:53:47,036 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2faa234e4e385b82c11337930631d2a1: 2023-05-05 15:53:47,036 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. 2023-05-05 15:53:47,036 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:53:47,037 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:53:47,037 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,037 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-05 15:53:47,037 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-05 15:53:47,038 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,039 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:47,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:47,039 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,039 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:47,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:47,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:47,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:47,042 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-05-05 15:53:47,042 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f6bf683[Count = 0] remaining members to acquire global barrier 2023-05-05 15:53:47,042 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-05-05 15:53:47,042 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,043 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,043 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,043 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,044 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-05 15:53:47,044 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-05 15:53:47,044 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-05 15:53:47,044 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,044 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:53:47,044 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:53:47,044 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-05 15:53:47,044 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-05 15:53:47,044 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-05 15:53:47,044 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:53:47,044 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:53:47,048 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-05 15:53:47,048 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:47,048 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-05 15:53:47,049 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-05 15:53:47,049 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-05 15:53:47,049 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,049 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:47,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,049 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:47,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:47,049 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-05 15:53:47,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:47,050 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-05 15:53:47,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:47,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:47,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:47,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,42799,1683302022977': 2023-05-05 15:53:47,056 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-05-05 15:53:47,056 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-05-05 15:53:47,056 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:53:47,056 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:53:47,056 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,056 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:53:47,058 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,058 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,058 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:47,058 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,058 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:47,058 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,058 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:47,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:47,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:47,059 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:47,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:47,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:47,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:47,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:47,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:47,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:47,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,059 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,060 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:47,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,063 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:47,063 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,064 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,067 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,067 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,067 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:47,067 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,067 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:47,068 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:47,068 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:47,068 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:47,067 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:47,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:47,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:47,067 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:47,068 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-05-05 15:53:47,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:47,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:47,068 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:47,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:47,068 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:53:47,068 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:47,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:47,068 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,069 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,069 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,069 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,069 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,069 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:47,069 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-05-05 15:53:47,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:47,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:47,069 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:53:47,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:47,169 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:53:47,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:53:47,195 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:53:47,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-05-05 15:53:47,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,197 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-05-05 15:53:47,197 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:47,198 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-05-05 15:53:47,198 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-05-05 15:53:47,199 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,199 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,201 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:47,201 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:47,201 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:47,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:47,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:47,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:47,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:47,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:47,201 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:47,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,202 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,202 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:53:47,203 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,203 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,203 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:47,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:47,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:47,203 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,203 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,203 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,204 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-05 15:53:47,204 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,204 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-05 15:53:47,204 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:47,204 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-05 15:53:47,205 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-05 15:53:47,204 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-05 15:53:47,205 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-05 15:53:47,205 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,205 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,205 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-05 15:53:47,205 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-05 15:53:47,205 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:47,205 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:47,206 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-05 15:53:47,205 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-05 15:53:47,206 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-05 15:53:47,206 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-05 15:53:47,206 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-05 15:53:47,207 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:53:47,207 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:53:47,207 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. 2023-05-05 15:53:47,207 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. started... 2023-05-05 15:53:47,207 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2faa234e4e385b82c11337930631d2a1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:53:47,208 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,209 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,209 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,209 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:47,210 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,210 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:47,230 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf1/8dab1931c3534993ba0f8cd43acfa5f2 2023-05-05 15:53:47,257 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf2/7414dd4b803f45fca86c7fa465350149 2023-05-05 15:53:47,267 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf1/8dab1931c3534993ba0f8cd43acfa5f2 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/8dab1931c3534993ba0f8cd43acfa5f2 2023-05-05 15:53:47,274 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/8dab1931c3534993ba0f8cd43acfa5f2, entries=10, sequenceid=45, filesize=5.0 K 2023-05-05 15:53:47,275 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf2/7414dd4b803f45fca86c7fa465350149 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/7414dd4b803f45fca86c7fa465350149 2023-05-05 15:53:47,283 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/7414dd4b803f45fca86c7fa465350149, entries=10, sequenceid=45, filesize=5.0 K 2023-05-05 15:53:47,284 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2faa234e4e385b82c11337930631d2a1 in 77ms, sequenceid=45, compaction requested=true 2023-05-05 15:53:47,285 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2faa234e4e385b82c11337930631d2a1: 2023-05-05 15:53:47,285 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. 2023-05-05 15:53:47,285 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:53:47,287 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:53:47,287 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,287 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-05 15:53:47,287 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-05 15:53:47,293 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,293 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:47,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:47,294 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,294 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:47,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:47,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:47,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:47,296 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-05-05 15:53:47,296 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@39e56af5[Count = 0] remaining members to acquire global barrier 2023-05-05 15:53:47,296 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-05-05 15:53:47,296 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,298 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,298 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,298 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,298 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-05 15:53:47,298 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,298 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:53:47,298 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-05 15:53:47,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,298 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-05 15:53:47,298 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-05 15:53:47,298 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:53:47,298 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-05 15:53:47,298 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-05 15:53:47,298 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:53:47,298 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:53:47,300 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-05 15:53:47,300 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:47,300 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-05 15:53:47,302 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-05 15:53:47,302 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-05 15:53:47,302 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,302 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:47,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:47,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:47,302 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:47,302 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-05 15:53:47,303 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-05 15:53:47,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:47,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:47,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:47,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,42799,1683302022977': 2023-05-05 15:53:47,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-05-05 15:53:47,307 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-05-05 15:53:47,307 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:53:47,307 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:53:47,308 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,308 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:53:47,309 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,309 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,310 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:47,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:47,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,310 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:47,309 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:47,309 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,310 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:47,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:47,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:47,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:47,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:47,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:47,311 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:47,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:47,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,311 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:47,312 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,314 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:47,314 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,314 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,318 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,318 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,318 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:47,318 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:47,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:47,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:47,318 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:47,318 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:47,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:47,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:47,318 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:47,318 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,318 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:47,318 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-05-05 15:53:47,319 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:53:47,318 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:47,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:47,318 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:47,320 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:47,320 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:47,320 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-05-05 15:53:47,320 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,320 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:53:47,320 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,320 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,320 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,320 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:47,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:47,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:47,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:47,420 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:53:47,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:53:47,446 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:53:47,448 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-05-05 15:53:47,448 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,448 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-05-05 15:53:47,449 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:47,449 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-05-05 15:53:47,449 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-05-05 15:53:47,450 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,450 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,451 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:47,451 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:47,451 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:47,451 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:47,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:47,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:47,452 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:47,452 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:53:47,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:47,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:47,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,453 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,453 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,453 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:47,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:47,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:47,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,453 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,453 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,453 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,453 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-05 15:53:47,454 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-05 15:53:47,454 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,454 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,454 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-05 15:53:47,454 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-05 15:53:47,454 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:47,454 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-05 15:53:47,454 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-05 15:53:47,455 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,454 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:47,455 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-05 15:53:47,455 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-05 15:53:47,455 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-05 15:53:47,455 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:47,455 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-05 15:53:47,457 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-05 15:53:47,457 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-05 15:53:47,457 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. 2023-05-05 15:53:47,457 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:53:47,457 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. started... 2023-05-05 15:53:47,457 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:53:47,457 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-05 15:53:47,457 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2faa234e4e385b82c11337930631d2a1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:53:47,460 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,460 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,460 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,461 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:47,461 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,461 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:47,479 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf1/2f51cb98d16e4678ba9160353e4141ce 2023-05-05 15:53:47,506 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf2/bc69b5ee5e594dbd8f6fcdf67defe662 2023-05-05 15:53:47,515 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf1/2f51cb98d16e4678ba9160353e4141ce as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/2f51cb98d16e4678ba9160353e4141ce 2023-05-05 15:53:47,522 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/2f51cb98d16e4678ba9160353e4141ce, entries=10, sequenceid=58, filesize=5.0 K 2023-05-05 15:53:47,523 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf2/bc69b5ee5e594dbd8f6fcdf67defe662 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/bc69b5ee5e594dbd8f6fcdf67defe662 2023-05-05 15:53:47,530 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/bc69b5ee5e594dbd8f6fcdf67defe662, entries=10, sequenceid=58, filesize=5.0 K 2023-05-05 15:53:47,532 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2faa234e4e385b82c11337930631d2a1 in 75ms, sequenceid=58, compaction requested=true 2023-05-05 15:53:47,532 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2faa234e4e385b82c11337930631d2a1: 2023-05-05 15:53:47,532 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. 2023-05-05 15:53:47,532 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:53:47,533 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:53:47,533 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,533 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-05 15:53:47,533 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-05 15:53:47,535 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,535 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:47,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:47,536 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,536 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:47,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:47,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:47,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:47,538 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-05-05 15:53:47,538 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b8add7c[Count = 0] remaining members to acquire global barrier 2023-05-05 15:53:47,538 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-05-05 15:53:47,538 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,539 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,539 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,539 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,539 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-05 15:53:47,540 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-05 15:53:47,540 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-05 15:53:47,540 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,540 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:53:47,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,540 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:53:47,540 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-05 15:53:47,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,540 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:53:47,540 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-05 15:53:47,540 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-05 15:53:47,540 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:53:47,542 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-05 15:53:47,542 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:47,542 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-05 15:53:47,543 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,543 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-05 15:53:47,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,543 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:47,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:47,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:47,543 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-05 15:53:47,544 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-05 15:53:47,544 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:47,545 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-05 15:53:47,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:47,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:47,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:47,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,42799,1683302022977': 2023-05-05 15:53:47,549 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-05-05 15:53:47,549 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-05-05 15:53:47,549 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:53:47,549 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:53:47,549 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,549 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:53:47,551 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,551 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,551 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,551 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:47,551 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,551 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:47,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,552 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:47,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:47,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:47,552 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:47,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:47,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:47,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:47,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:47,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:47,553 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:47,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,553 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:47,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,555 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:47,555 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,556 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,562 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,563 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,563 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:47,563 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:47,563 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:47,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:47,563 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:47,563 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:47,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:47,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:47,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:47,563 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,563 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:47,563 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:47,563 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:47,563 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-05-05 15:53:47,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:47,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:47,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:47,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:47,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:47,564 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:53:47,563 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:47,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:47,565 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,565 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,565 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,565 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,565 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,565 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,565 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-05-05 15:53:47,565 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:53:47,666 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:53:47,667 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:53:47,689 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:53:47,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-05-05 15:53:47,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,692 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-05-05 15:53:47,692 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:47,693 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-05-05 15:53:47,693 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-05-05 15:53:47,693 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,693 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,695 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,695 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:47,695 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:47,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:47,695 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:47,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:47,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:47,695 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,695 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:53:47,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:47,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:47,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:47,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,696 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,696 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,696 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:47,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,696 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:47,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,697 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:47,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,697 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-05 15:53:47,697 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,697 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-05 15:53:47,697 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,697 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,697 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-05 15:53:47,697 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-05 15:53:47,697 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:47,697 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-05 15:53:47,698 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-05 15:53:47,698 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,698 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-05 15:53:47,698 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-05 15:53:47,698 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:47,698 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-05 15:53:47,698 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-05 15:53:47,697 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:47,699 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-05 15:53:47,699 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:53:47,699 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:53:47,699 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. 2023-05-05 15:53:47,699 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-05 15:53:47,699 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. started... 2023-05-05 15:53:47,699 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-05 15:53:47,699 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2faa234e4e385b82c11337930631d2a1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:53:47,701 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,701 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,701 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:47,704 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,705 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,705 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:47,724 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf1/2089f9f8e89a452ab615acf8f64ee9a7 2023-05-05 15:53:47,745 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf2/863ba511c93d45cfaeb4f9a4e1720d66 2023-05-05 15:53:47,754 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf1/2089f9f8e89a452ab615acf8f64ee9a7 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/2089f9f8e89a452ab615acf8f64ee9a7 2023-05-05 15:53:47,761 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/2089f9f8e89a452ab615acf8f64ee9a7, entries=10, sequenceid=71, filesize=5.0 K 2023-05-05 15:53:47,763 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf2/863ba511c93d45cfaeb4f9a4e1720d66 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/863ba511c93d45cfaeb4f9a4e1720d66 2023-05-05 15:53:47,770 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/863ba511c93d45cfaeb4f9a4e1720d66, entries=10, sequenceid=71, filesize=5.0 K 2023-05-05 15:53:47,771 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2faa234e4e385b82c11337930631d2a1 in 72ms, sequenceid=71, compaction requested=true 2023-05-05 15:53:47,771 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2faa234e4e385b82c11337930631d2a1: 2023-05-05 15:53:47,771 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. 2023-05-05 15:53:47,772 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:53:47,772 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:53:47,772 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,772 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-05 15:53:47,772 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-05 15:53:47,774 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,774 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:47,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:47,775 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,775 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:47,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:47,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:47,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:47,777 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-05-05 15:53:47,777 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-05-05 15:53:47,777 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5397a5bf[Count = 0] remaining members to acquire global barrier 2023-05-05 15:53:47,777 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,778 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,778 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,778 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,779 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-05 15:53:47,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,779 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-05 15:53:47,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,779 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:53:47,779 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-05 15:53:47,779 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-05 15:53:47,779 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,779 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:53:47,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,779 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:53:47,779 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-05 15:53:47,779 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-05 15:53:47,779 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:53:47,781 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-05 15:53:47,781 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:47,781 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-05 15:53:47,782 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-05 15:53:47,782 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,782 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:47,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:47,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:47,782 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-05 15:53:47,783 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:47,782 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-05 15:53:47,783 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-05 15:53:47,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:47,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:47,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:47,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,42799,1683302022977': 2023-05-05 15:53:47,787 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-05-05 15:53:47,787 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-05-05 15:53:47,787 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:53:47,787 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:53:47,787 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,787 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:53:47,789 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,789 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,789 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,789 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:47,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,789 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:47,789 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:47,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:47,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:47,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:47,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,790 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,790 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:47,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:47,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:47,790 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:47,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:47,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:47,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,791 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:47,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,793 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:47,793 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,793 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,796 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,796 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,796 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:47,796 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:47,796 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,796 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:47,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:47,797 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:47,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:47,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:47,797 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-05-05 15:53:47,797 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:53:47,796 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:47,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:47,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:47,797 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:47,797 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:47,797 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:47,797 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:47,797 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-05-05 15:53:47,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:47,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:47,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:47,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:47,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:47,797 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:47,798 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:53:47,798 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,799 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:47,799 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,799 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,799 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-05 15:53:47,899 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:53:47,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:53:47,922 DEBUG [Listener at localhost/35117] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,929 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:47,930 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:47,930 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:47,930 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:47,937 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=2faa234e4e385b82c11337930631d2a1, UNASSIGN}] 2023-05-05 15:53:47,939 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=2faa234e4e385b82c11337930631d2a1, UNASSIGN 2023-05-05 15:53:47,940 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=2faa234e4e385b82c11337930631d2a1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:47,940 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1683302027940"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302027940"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302027940"}]},"ts":"1683302027940"} 2023-05-05 15:53:47,942 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 2faa234e4e385b82c11337930631d2a1, server=jenkins-hbase4.apache.org,42799,1683302022977}] 2023-05-05 15:53:48,103 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,103 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2faa234e4e385b82c11337930631d2a1, disabling compactions & flushes 2023-05-05 15:53:48,103 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. 2023-05-05 15:53:48,103 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. 2023-05-05 15:53:48,103 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. after waiting 0 ms 2023-05-05 15:53:48,103 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. 2023-05-05 15:53:48,103 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 2faa234e4e385b82c11337930631d2a1 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-05 15:53:48,129 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf1/523fa9e6142b468d94edaeb8d9b6aeae 2023-05-05 15:53:48,161 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf2/8d3b8f1928d8481997156b461034e9fa 2023-05-05 15:53:48,168 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf1/523fa9e6142b468d94edaeb8d9b6aeae as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/523fa9e6142b468d94edaeb8d9b6aeae 2023-05-05 15:53:48,175 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/523fa9e6142b468d94edaeb8d9b6aeae, entries=8, sequenceid=82, filesize=5.0 K 2023-05-05 15:53:48,176 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.tmp/cf2/8d3b8f1928d8481997156b461034e9fa as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/8d3b8f1928d8481997156b461034e9fa 2023-05-05 15:53:48,184 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/8d3b8f1928d8481997156b461034e9fa, entries=8, sequenceid=82, filesize=5.0 K 2023-05-05 15:53:48,185 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 2faa234e4e385b82c11337930631d2a1 in 82ms, sequenceid=82, compaction requested=true 2023-05-05 15:53:48,209 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-05 15:53:48,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:53:48,211 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. 2023-05-05 15:53:48,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2faa234e4e385b82c11337930631d2a1: 2023-05-05 15:53:48,214 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,214 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=2faa234e4e385b82c11337930631d2a1, regionState=CLOSED 2023-05-05 15:53:48,215 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1683302028214"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302028214"}]},"ts":"1683302028214"} 2023-05-05 15:53:48,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-05-05 15:53:48,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 2faa234e4e385b82c11337930631d2a1, server=jenkins-hbase4.apache.org,42799,1683302022977 in 276 msec 2023-05-05 15:53:48,222 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-05-05 15:53:48,223 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=2faa234e4e385b82c11337930631d2a1, UNASSIGN in 283 msec 2023-05-05 15:53:48,246 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=2faa234e4e385b82c11337930631d2a1, threads=14 2023-05-05 15:53:48,247 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/1903627febdd4806832679268b98ef95 for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,247 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/33cfd2a14784445ba210b3c709d236ec for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,248 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/4991bfadfb32442e904704119d917f44 for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,248 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/7414dd4b803f45fca86c7fa465350149 for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,248 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/863ba511c93d45cfaeb4f9a4e1720d66 for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,248 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/8d3b8f1928d8481997156b461034e9fa for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,250 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/bc69b5ee5e594dbd8f6fcdf67defe662 for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,250 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/043c3df5b6d24e3cbf5e78974ea62314 for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,250 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/17846580efc44fc9b1bbdfb8e6f5a0de for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,250 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/2089f9f8e89a452ab615acf8f64ee9a7 for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,251 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/2f51cb98d16e4678ba9160353e4141ce for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,251 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/523fa9e6142b468d94edaeb8d9b6aeae for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,252 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/8dab1931c3534993ba0f8cd43acfa5f2 for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,253 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/d69a911d9c294c03b7032ffbd281155f for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,293 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/8d3b8f1928d8481997156b461034e9fa, top=true 2023-05-05 15:53:48,296 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/2089f9f8e89a452ab615acf8f64ee9a7, top=true 2023-05-05 15:53:48,301 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/523fa9e6142b468d94edaeb8d9b6aeae, top=true 2023-05-05 15:53:48,304 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/863ba511c93d45cfaeb4f9a4e1720d66, top=true 2023-05-05 15:53:48,312 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/2f51cb98d16e4678ba9160353e4141ce, top=true 2023-05-05 15:53:48,374 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.splits/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-2089f9f8e89a452ab615acf8f64ee9a7 for child: 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9, parent: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,374 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/2089f9f8e89a452ab615acf8f64ee9a7 for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,375 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/bc69b5ee5e594dbd8f6fcdf67defe662, top=true 2023-05-05 15:53:48,378 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.splits/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-8d3b8f1928d8481997156b461034e9fa for child: 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9, parent: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,378 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/8d3b8f1928d8481997156b461034e9fa for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,392 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.splits/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-863ba511c93d45cfaeb4f9a4e1720d66 for child: 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9, parent: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,392 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/863ba511c93d45cfaeb4f9a4e1720d66 for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,398 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.splits/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-2f51cb98d16e4678ba9160353e4141ce for child: 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9, parent: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,398 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/2f51cb98d16e4678ba9160353e4141ce for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,400 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.splits/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-523fa9e6142b468d94edaeb8d9b6aeae for child: 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9, parent: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,400 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/523fa9e6142b468d94edaeb8d9b6aeae for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,419 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/.splits/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-bc69b5ee5e594dbd8f6fcdf67defe662 for child: 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9, parent: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,419 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/bc69b5ee5e594dbd8f6fcdf67defe662 for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,471 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/4991bfadfb32442e904704119d917f44 for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,517 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/7414dd4b803f45fca86c7fa465350149 for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,517 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/17846580efc44fc9b1bbdfb8e6f5a0de for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,517 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/043c3df5b6d24e3cbf5e78974ea62314 for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,518 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/33cfd2a14784445ba210b3c709d236ec for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,522 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/d69a911d9c294c03b7032ffbd281155f for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,525 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/8dab1931c3534993ba0f8cd43acfa5f2 for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,821 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/1903627febdd4806832679268b98ef95 for region: 2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:48,821 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 2faa234e4e385b82c11337930631d2a1 Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-05-05 15:53:48,889 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-05 15:53:48,892 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-05 15:53:48,894 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1683302028894"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1683302028894"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1683302028894"}]},"ts":"1683302028894"} 2023-05-05 15:53:48,895 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683302028894"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302028894"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302028894"}]},"ts":"1683302028894"} 2023-05-05 15:53:48,895 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683302028894"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302028894"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302028894"}]},"ts":"1683302028894"} 2023-05-05 15:53:48,943 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e24ba92d7211d01fc6da0d90d6b8b4e1, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9, ASSIGN}] 2023-05-05 15:53:48,944 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9, ASSIGN 2023-05-05 15:53:48,945 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42799,1683302022977; forceNewPlan=false, retain=false 2023-05-05 15:53:48,948 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e24ba92d7211d01fc6da0d90d6b8b4e1, ASSIGN 2023-05-05 15:53:48,949 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e24ba92d7211d01fc6da0d90d6b8b4e1, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42799,1683302022977; forceNewPlan=false, retain=false 2023-05-05 15:53:49,095 INFO [jenkins-hbase4:44213] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-05 15:53:49,097 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=e24ba92d7211d01fc6da0d90d6b8b4e1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:49,097 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683302029097"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302029097"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302029097"}]},"ts":"1683302029097"} 2023-05-05 15:53:49,099 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure e24ba92d7211d01fc6da0d90d6b8b4e1, server=jenkins-hbase4.apache.org,42799,1683302022977}] 2023-05-05 15:53:49,103 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:49,103 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683302029103"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302029103"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302029103"}]},"ts":"1683302029103"} 2023-05-05 15:53:49,106 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9, server=jenkins-hbase4.apache.org,42799,1683302022977}] 2023-05-05 15:53:49,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1. 2023-05-05 15:53:49,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e24ba92d7211d01fc6da0d90d6b8b4e1, NAME => 'testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1.', STARTKEY => '', ENDKEY => '26'} 2023-05-05 15:53:49,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:53:49,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter e24ba92d7211d01fc6da0d90d6b8b4e1 2023-05-05 15:53:49,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:53:49,257 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e24ba92d7211d01fc6da0d90d6b8b4e1 2023-05-05 15:53:49,257 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e24ba92d7211d01fc6da0d90d6b8b4e1 2023-05-05 15:53:49,258 INFO [StoreOpener-e24ba92d7211d01fc6da0d90d6b8b4e1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e24ba92d7211d01fc6da0d90d6b8b4e1 2023-05-05 15:53:49,259 DEBUG [StoreOpener-e24ba92d7211d01fc6da0d90d6b8b4e1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1 2023-05-05 15:53:49,259 DEBUG [StoreOpener-e24ba92d7211d01fc6da0d90d6b8b4e1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1 2023-05-05 15:53:49,260 INFO [StoreOpener-e24ba92d7211d01fc6da0d90d6b8b4e1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e24ba92d7211d01fc6da0d90d6b8b4e1 columnFamilyName cf1 2023-05-05 15:53:49,292 DEBUG [StoreOpener-e24ba92d7211d01fc6da0d90d6b8b4e1-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1/043c3df5b6d24e3cbf5e78974ea62314.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/043c3df5b6d24e3cbf5e78974ea62314-bottom 2023-05-05 15:53:49,300 DEBUG [StoreOpener-e24ba92d7211d01fc6da0d90d6b8b4e1-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1/17846580efc44fc9b1bbdfb8e6f5a0de.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/17846580efc44fc9b1bbdfb8e6f5a0de-bottom 2023-05-05 15:53:49,306 DEBUG [StoreOpener-e24ba92d7211d01fc6da0d90d6b8b4e1-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1/8dab1931c3534993ba0f8cd43acfa5f2.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/8dab1931c3534993ba0f8cd43acfa5f2-bottom 2023-05-05 15:53:49,314 DEBUG [StoreOpener-e24ba92d7211d01fc6da0d90d6b8b4e1-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1/d69a911d9c294c03b7032ffbd281155f.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/d69a911d9c294c03b7032ffbd281155f-bottom 2023-05-05 15:53:49,314 INFO [StoreOpener-e24ba92d7211d01fc6da0d90d6b8b4e1-1] regionserver.HStore(310): Store=e24ba92d7211d01fc6da0d90d6b8b4e1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:53:49,315 INFO [StoreOpener-e24ba92d7211d01fc6da0d90d6b8b4e1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e24ba92d7211d01fc6da0d90d6b8b4e1 2023-05-05 15:53:49,316 DEBUG [StoreOpener-e24ba92d7211d01fc6da0d90d6b8b4e1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2 2023-05-05 15:53:49,316 DEBUG [StoreOpener-e24ba92d7211d01fc6da0d90d6b8b4e1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2 2023-05-05 15:53:49,317 INFO [StoreOpener-e24ba92d7211d01fc6da0d90d6b8b4e1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e24ba92d7211d01fc6da0d90d6b8b4e1 columnFamilyName cf2 2023-05-05 15:53:49,333 DEBUG [StoreOpener-e24ba92d7211d01fc6da0d90d6b8b4e1-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2/1903627febdd4806832679268b98ef95.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/1903627febdd4806832679268b98ef95-bottom 2023-05-05 15:53:49,341 DEBUG [StoreOpener-e24ba92d7211d01fc6da0d90d6b8b4e1-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2/33cfd2a14784445ba210b3c709d236ec.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/33cfd2a14784445ba210b3c709d236ec-bottom 2023-05-05 15:53:49,348 DEBUG [StoreOpener-e24ba92d7211d01fc6da0d90d6b8b4e1-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2/4991bfadfb32442e904704119d917f44.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/4991bfadfb32442e904704119d917f44-bottom 2023-05-05 15:53:49,355 DEBUG [StoreOpener-e24ba92d7211d01fc6da0d90d6b8b4e1-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2/7414dd4b803f45fca86c7fa465350149.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/7414dd4b803f45fca86c7fa465350149-bottom 2023-05-05 15:53:49,355 INFO [StoreOpener-e24ba92d7211d01fc6da0d90d6b8b4e1-1] regionserver.HStore(310): Store=e24ba92d7211d01fc6da0d90d6b8b4e1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:53:49,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1 2023-05-05 15:53:49,358 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1 2023-05-05 15:53:49,360 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-05 15:53:49,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e24ba92d7211d01fc6da0d90d6b8b4e1 2023-05-05 15:53:49,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e24ba92d7211d01fc6da0d90d6b8b4e1; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71364240, jitterRate=0.06341004371643066}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-05 15:53:49,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e24ba92d7211d01fc6da0d90d6b8b4e1: 2023-05-05 15:53:49,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1., pid=17, masterSystemTime=1683302029251 2023-05-05 15:53:49,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:53:49,368 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-05 15:53:49,368 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-05-05 15:53:49,368 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-05-05 15:53:49,370 INFO [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1. 2023-05-05 15:53:49,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1. 2023-05-05 15:53:49,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1. 2023-05-05 15:53:49,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9. 2023-05-05 15:53:49,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9, NAME => 'testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.', STARTKEY => '26', ENDKEY => ''} 2023-05-05 15:53:49,370 INFO [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1. 2023-05-05 15:53:49,370 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.HStore(1912): e24ba92d7211d01fc6da0d90d6b8b4e1/cf2 is initiating minor compaction (all files) 2023-05-05 15:53:49,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:53:49,371 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.HStore(1912): e24ba92d7211d01fc6da0d90d6b8b4e1/cf1 is initiating minor compaction (all files) 2023-05-05 15:53:49,371 INFO [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.HRegion(2259): Starting compaction of e24ba92d7211d01fc6da0d90d6b8b4e1/cf2 in testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1. 2023-05-05 15:53:49,371 INFO [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of e24ba92d7211d01fc6da0d90d6b8b4e1/cf1 in testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1. 2023-05-05 15:53:49,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:49,371 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=e24ba92d7211d01fc6da0d90d6b8b4e1, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:49,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:53:49,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:49,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:49,372 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683302029371"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302029371"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302029371"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302029371"}]},"ts":"1683302029371"} 2023-05-05 15:53:49,372 INFO [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1/d69a911d9c294c03b7032ffbd281155f.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/d69a911d9c294c03b7032ffbd281155f-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1/043c3df5b6d24e3cbf5e78974ea62314.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/043c3df5b6d24e3cbf5e78974ea62314-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1/17846580efc44fc9b1bbdfb8e6f5a0de.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/17846580efc44fc9b1bbdfb8e6f5a0de-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1/8dab1931c3534993ba0f8cd43acfa5f2.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/8dab1931c3534993ba0f8cd43acfa5f2-bottom] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/.tmp, totalSize=19.9 K 2023-05-05 15:53:49,372 INFO [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2/33cfd2a14784445ba210b3c709d236ec.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/33cfd2a14784445ba210b3c709d236ec-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2/4991bfadfb32442e904704119d917f44.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/4991bfadfb32442e904704119d917f44-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2/1903627febdd4806832679268b98ef95.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/1903627febdd4806832679268b98ef95-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2/7414dd4b803f45fca86c7fa465350149.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/7414dd4b803f45fca86c7fa465350149-bottom] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/.tmp, totalSize=19.9 K 2023-05-05 15:53:49,374 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] compactions.Compactor(207): Compacting 33cfd2a14784445ba210b3c709d236ec.2faa234e4e385b82c11337930631d2a1, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683302026274 2023-05-05 15:53:49,374 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] compactions.Compactor(207): Compacting d69a911d9c294c03b7032ffbd281155f.2faa234e4e385b82c11337930631d2a1, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683302026274 2023-05-05 15:53:49,376 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] compactions.Compactor(207): Compacting 4991bfadfb32442e904704119d917f44.2faa234e4e385b82c11337930631d2a1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683302026655 2023-05-05 15:53:49,376 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] compactions.Compactor(207): Compacting 043c3df5b6d24e3cbf5e78974ea62314.2faa234e4e385b82c11337930631d2a1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683302026655 2023-05-05 15:53:49,377 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] compactions.Compactor(207): Compacting 1903627febdd4806832679268b98ef95.2faa234e4e385b82c11337930631d2a1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683302026924 2023-05-05 15:53:49,377 INFO [StoreOpener-7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:49,377 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] compactions.Compactor(207): Compacting 17846580efc44fc9b1bbdfb8e6f5a0de.2faa234e4e385b82c11337930631d2a1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683302026924 2023-05-05 15:53:49,378 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] compactions.Compactor(207): Compacting 7414dd4b803f45fca86c7fa465350149.2faa234e4e385b82c11337930631d2a1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683302027172 2023-05-05 15:53:49,378 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] compactions.Compactor(207): Compacting 8dab1931c3534993ba0f8cd43acfa5f2.2faa234e4e385b82c11337930631d2a1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683302027172 2023-05-05 15:53:49,379 DEBUG [StoreOpener-7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1 2023-05-05 15:53:49,379 DEBUG [StoreOpener-7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1 2023-05-05 15:53:49,380 INFO [StoreOpener-7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 columnFamilyName cf1 2023-05-05 15:53:49,380 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-05-05 15:53:49,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure e24ba92d7211d01fc6da0d90d6b8b4e1, server=jenkins-hbase4.apache.org,42799,1683302022977 in 278 msec 2023-05-05 15:53:49,384 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e24ba92d7211d01fc6da0d90d6b8b4e1, ASSIGN in 438 msec 2023-05-05 15:53:49,404 DEBUG [StoreOpener-7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/043c3df5b6d24e3cbf5e78974ea62314.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/043c3df5b6d24e3cbf5e78974ea62314-top 2023-05-05 15:53:49,413 DEBUG [StoreOpener-7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/17846580efc44fc9b1bbdfb8e6f5a0de.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/17846580efc44fc9b1bbdfb8e6f5a0de-top 2023-05-05 15:53:49,415 INFO [RS:1;jenkins-hbase4:42799-longCompactions-0] throttle.PressureAwareThroughputController(145): e24ba92d7211d01fc6da0d90d6b8b4e1#cf2#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:53:49,416 INFO [RS:1;jenkins-hbase4:42799-shortCompactions-0] throttle.PressureAwareThroughputController(145): e24ba92d7211d01fc6da0d90d6b8b4e1#cf1#compaction#15 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:53:49,432 DEBUG [StoreOpener-7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/8dab1931c3534993ba0f8cd43acfa5f2.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/8dab1931c3534993ba0f8cd43acfa5f2-top 2023-05-05 15:53:49,446 DEBUG [StoreOpener-7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/d69a911d9c294c03b7032ffbd281155f.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/d69a911d9c294c03b7032ffbd281155f-top 2023-05-05 15:53:49,451 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/.tmp/cf2/1097bb8b308b40dd8e397344d9dbef44 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2/1097bb8b308b40dd8e397344d9dbef44 2023-05-05 15:53:49,453 DEBUG [StoreOpener-7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-2089f9f8e89a452ab615acf8f64ee9a7 2023-05-05 15:53:49,456 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/.tmp/cf1/09271e080258433fb610e3de0a026492 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1/09271e080258433fb610e3de0a026492 2023-05-05 15:53:49,462 DEBUG [StoreOpener-7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-2f51cb98d16e4678ba9160353e4141ce 2023-05-05 15:53:49,468 DEBUG [StoreOpener-7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-523fa9e6142b468d94edaeb8d9b6aeae 2023-05-05 15:53:49,468 INFO [StoreOpener-7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9-1] regionserver.HStore(310): Store=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:53:49,468 INFO [StoreOpener-7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:49,470 DEBUG [StoreOpener-7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2 2023-05-05 15:53:49,470 DEBUG [StoreOpener-7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2 2023-05-05 15:53:49,470 INFO [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in e24ba92d7211d01fc6da0d90d6b8b4e1/cf1 of e24ba92d7211d01fc6da0d90d6b8b4e1 into 09271e080258433fb610e3de0a026492(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:53:49,470 INFO [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in e24ba92d7211d01fc6da0d90d6b8b4e1/cf2 of e24ba92d7211d01fc6da0d90d6b8b4e1 into 1097bb8b308b40dd8e397344d9dbef44(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:53:49,471 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for e24ba92d7211d01fc6da0d90d6b8b4e1: 2023-05-05 15:53:49,471 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for e24ba92d7211d01fc6da0d90d6b8b4e1: 2023-05-05 15:53:49,471 INFO [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1., storeName=e24ba92d7211d01fc6da0d90d6b8b4e1/cf1, priority=12, startTime=1683302029367; duration=0sec 2023-05-05 15:53:49,471 INFO [StoreOpener-7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 columnFamilyName cf2 2023-05-05 15:53:49,472 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:53:49,471 INFO [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1., storeName=e24ba92d7211d01fc6da0d90d6b8b4e1/cf2, priority=12, startTime=1683302029368; duration=0sec 2023-05-05 15:53:49,473 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:53:49,486 DEBUG [StoreOpener-7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/1903627febdd4806832679268b98ef95.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/1903627febdd4806832679268b98ef95-top 2023-05-05 15:53:49,490 DEBUG [StoreOpener-7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/33cfd2a14784445ba210b3c709d236ec.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/33cfd2a14784445ba210b3c709d236ec-top 2023-05-05 15:53:49,496 DEBUG [StoreOpener-7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/4991bfadfb32442e904704119d917f44.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/4991bfadfb32442e904704119d917f44-top 2023-05-05 15:53:49,502 DEBUG [StoreOpener-7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/7414dd4b803f45fca86c7fa465350149.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/7414dd4b803f45fca86c7fa465350149-top 2023-05-05 15:53:49,507 DEBUG [StoreOpener-7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-863ba511c93d45cfaeb4f9a4e1720d66 2023-05-05 15:53:49,511 DEBUG [StoreOpener-7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-8d3b8f1928d8481997156b461034e9fa 2023-05-05 15:53:49,518 DEBUG [StoreOpener-7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-bc69b5ee5e594dbd8f6fcdf67defe662 2023-05-05 15:53:49,518 INFO [StoreOpener-7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9-1] regionserver.HStore(310): Store=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:53:49,519 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:49,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:49,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-05 15:53:49,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:49,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66971329, jitterRate=-0.0020494312047958374}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-05 15:53:49,526 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9: 2023-05-05 15:53:49,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9., pid=18, masterSystemTime=1683302029251 2023-05-05 15:53:49,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:53:49,529 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-05 15:53:49,529 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-05 15:53:49,529 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-05 15:53:49,529 INFO [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9. 2023-05-05 15:53:49,529 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.HStore(1912): 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1 is initiating minor compaction (all files) 2023-05-05 15:53:49,529 INFO [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9. 2023-05-05 15:53:49,529 INFO [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1 in testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9. 2023-05-05 15:53:49,529 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.HStore(1912): 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2 is initiating minor compaction (all files) 2023-05-05 15:53:49,529 INFO [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2 in testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9. 2023-05-05 15:53:49,530 INFO [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/d69a911d9c294c03b7032ffbd281155f.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/d69a911d9c294c03b7032ffbd281155f-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/043c3df5b6d24e3cbf5e78974ea62314.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/043c3df5b6d24e3cbf5e78974ea62314-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/17846580efc44fc9b1bbdfb8e6f5a0de.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/17846580efc44fc9b1bbdfb8e6f5a0de-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/8dab1931c3534993ba0f8cd43acfa5f2.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/8dab1931c3534993ba0f8cd43acfa5f2-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-2f51cb98d16e4678ba9160353e4141ce, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-2089f9f8e89a452ab615acf8f64ee9a7, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-523fa9e6142b468d94edaeb8d9b6aeae] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/.tmp, totalSize=34.9 K 2023-05-05 15:53:49,530 INFO [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/33cfd2a14784445ba210b3c709d236ec.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/33cfd2a14784445ba210b3c709d236ec-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/4991bfadfb32442e904704119d917f44.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/4991bfadfb32442e904704119d917f44-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/1903627febdd4806832679268b98ef95.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/1903627febdd4806832679268b98ef95-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/7414dd4b803f45fca86c7fa465350149.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/7414dd4b803f45fca86c7fa465350149-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-bc69b5ee5e594dbd8f6fcdf67defe662, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-863ba511c93d45cfaeb4f9a4e1720d66, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-8d3b8f1928d8481997156b461034e9fa] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/.tmp, totalSize=34.9 K 2023-05-05 15:53:49,532 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] compactions.Compactor(207): Compacting d69a911d9c294c03b7032ffbd281155f.2faa234e4e385b82c11337930631d2a1, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683302026274 2023-05-05 15:53:49,532 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9. 2023-05-05 15:53:49,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9. 2023-05-05 15:53:49,534 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] compactions.Compactor(207): Compacting 33cfd2a14784445ba210b3c709d236ec.2faa234e4e385b82c11337930631d2a1, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683302026274 2023-05-05 15:53:49,534 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] compactions.Compactor(207): Compacting 043c3df5b6d24e3cbf5e78974ea62314.2faa234e4e385b82c11337930631d2a1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683302026655 2023-05-05 15:53:49,534 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] compactions.Compactor(207): Compacting 4991bfadfb32442e904704119d917f44.2faa234e4e385b82c11337930631d2a1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683302026655 2023-05-05 15:53:49,535 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:49,535 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] compactions.Compactor(207): Compacting 17846580efc44fc9b1bbdfb8e6f5a0de.2faa234e4e385b82c11337930631d2a1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683302026924 2023-05-05 15:53:49,535 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683302029535"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302029535"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302029535"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302029535"}]},"ts":"1683302029535"} 2023-05-05 15:53:49,537 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] compactions.Compactor(207): Compacting 1903627febdd4806832679268b98ef95.2faa234e4e385b82c11337930631d2a1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683302026924 2023-05-05 15:53:49,537 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] compactions.Compactor(207): Compacting 8dab1931c3534993ba0f8cd43acfa5f2.2faa234e4e385b82c11337930631d2a1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1683302027172 2023-05-05 15:53:49,538 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] compactions.Compactor(207): Compacting 7414dd4b803f45fca86c7fa465350149.2faa234e4e385b82c11337930631d2a1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1683302027172 2023-05-05 15:53:49,539 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-2f51cb98d16e4678ba9160353e4141ce, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683302027423 2023-05-05 15:53:49,539 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-bc69b5ee5e594dbd8f6fcdf67defe662, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683302027423 2023-05-05 15:53:49,539 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-863ba511c93d45cfaeb4f9a4e1720d66, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683302027668 2023-05-05 15:53:49,539 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-2089f9f8e89a452ab615acf8f64ee9a7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683302027668 2023-05-05 15:53:49,540 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-8d3b8f1928d8481997156b461034e9fa, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683302027902 2023-05-05 15:53:49,540 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-523fa9e6142b468d94edaeb8d9b6aeae, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683302027902 2023-05-05 15:53:49,544 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=16 2023-05-05 15:53:49,544 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9, server=jenkins-hbase4.apache.org,42799,1683302022977 in 433 msec 2023-05-05 15:53:49,548 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=12 2023-05-05 15:53:49,549 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9, ASSIGN in 601 msec 2023-05-05 15:53:49,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 in 1.6270 sec 2023-05-05 15:53:49,573 INFO [RS:1;jenkins-hbase4:42799-longCompactions-0] throttle.PressureAwareThroughputController(145): 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9#cf2#compaction#16 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:53:49,578 INFO [RS:1;jenkins-hbase4:42799-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9#cf1#compaction#17 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:53:49,591 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/.tmp/cf2/9a36f6eede8a43fd813495dc3632612c as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/9a36f6eede8a43fd813495dc3632612c 2023-05-05 15:53:49,595 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/.tmp/cf1/132020930e2e449ba169b11bcb82115d as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/132020930e2e449ba169b11bcb82115d 2023-05-05 15:53:49,600 INFO [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2 of 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 into 9a36f6eede8a43fd813495dc3632612c(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:53:49,602 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9: 2023-05-05 15:53:49,602 INFO [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9., storeName=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2, priority=9, startTime=1683302029529; duration=0sec 2023-05-05 15:53:49,602 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:53:49,608 INFO [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1 of 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 into 132020930e2e449ba169b11bcb82115d(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:53:49,608 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9: 2023-05-05 15:53:49,608 INFO [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9., storeName=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1, priority=9, startTime=1683302029528; duration=0sec 2023-05-05 15:53:49,608 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:53:49,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42799] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:45666 deadline: 1683302089681, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1. is not online on jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:49,816 DEBUG [Listener at localhost/35117] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-05-05 15:53:49,828 INFO [Listener at localhost/35117] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-05-05 15:53:49,830 DEBUG [Listener at localhost/35117] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-05-05 15:53:49,868 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-05 15:53:49,878 INFO [Listener at localhost/35117] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-05-05 15:53:49,882 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionUnevenDaughter 2023-05-05 15:53:49,889 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:49,894 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302029894"}]},"ts":"1683302029894"} 2023-05-05 15:53:49,896 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-05-05 15:53:49,898 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-05-05 15:53:49,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-05-05 15:53:49,900 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e24ba92d7211d01fc6da0d90d6b8b4e1, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9, UNASSIGN}] 2023-05-05 15:53:49,901 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9, UNASSIGN 2023-05-05 15:53:49,902 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:49,903 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683302029902"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302029902"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302029902"}]},"ts":"1683302029902"} 2023-05-05 15:53:49,905 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9, server=jenkins-hbase4.apache.org,42799,1683302022977}] 2023-05-05 15:53:49,909 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e24ba92d7211d01fc6da0d90d6b8b4e1, UNASSIGN 2023-05-05 15:53:49,910 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=e24ba92d7211d01fc6da0d90d6b8b4e1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:49,910 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683302029910"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302029910"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302029910"}]},"ts":"1683302029910"} 2023-05-05 15:53:49,912 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure e24ba92d7211d01fc6da0d90d6b8b4e1, server=jenkins-hbase4.apache.org,42799,1683302022977}] 2023-05-05 15:53:50,009 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-05-05 15:53:50,058 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close e24ba92d7211d01fc6da0d90d6b8b4e1 2023-05-05 15:53:50,059 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e24ba92d7211d01fc6da0d90d6b8b4e1, disabling compactions & flushes 2023-05-05 15:53:50,059 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1. 2023-05-05 15:53:50,059 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1. 2023-05-05 15:53:50,059 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1. after waiting 0 ms 2023-05-05 15:53:50,059 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1. 2023-05-05 15:53:50,065 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1/d69a911d9c294c03b7032ffbd281155f.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/d69a911d9c294c03b7032ffbd281155f-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1/043c3df5b6d24e3cbf5e78974ea62314.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/043c3df5b6d24e3cbf5e78974ea62314-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1/17846580efc44fc9b1bbdfb8e6f5a0de.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/17846580efc44fc9b1bbdfb8e6f5a0de-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1/8dab1931c3534993ba0f8cd43acfa5f2.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/8dab1931c3534993ba0f8cd43acfa5f2-bottom] to archive 2023-05-05 15:53:50,067 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-05 15:53:50,074 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1/d69a911d9c294c03b7032ffbd281155f.2faa234e4e385b82c11337930631d2a1 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1/d69a911d9c294c03b7032ffbd281155f.2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:50,076 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1/043c3df5b6d24e3cbf5e78974ea62314.2faa234e4e385b82c11337930631d2a1 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1/043c3df5b6d24e3cbf5e78974ea62314.2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:50,079 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1/17846580efc44fc9b1bbdfb8e6f5a0de.2faa234e4e385b82c11337930631d2a1 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1/17846580efc44fc9b1bbdfb8e6f5a0de.2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:50,081 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1/8dab1931c3534993ba0f8cd43acfa5f2.2faa234e4e385b82c11337930631d2a1 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1/8dab1931c3534993ba0f8cd43acfa5f2.2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:50,118 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2/33cfd2a14784445ba210b3c709d236ec.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/33cfd2a14784445ba210b3c709d236ec-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2/4991bfadfb32442e904704119d917f44.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/4991bfadfb32442e904704119d917f44-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2/1903627febdd4806832679268b98ef95.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/1903627febdd4806832679268b98ef95-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2/7414dd4b803f45fca86c7fa465350149.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/7414dd4b803f45fca86c7fa465350149-bottom] to archive 2023-05-05 15:53:50,120 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-05 15:53:50,123 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2/33cfd2a14784445ba210b3c709d236ec.2faa234e4e385b82c11337930631d2a1 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2/33cfd2a14784445ba210b3c709d236ec.2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:50,126 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2/4991bfadfb32442e904704119d917f44.2faa234e4e385b82c11337930631d2a1 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2/4991bfadfb32442e904704119d917f44.2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:50,128 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2/1903627febdd4806832679268b98ef95.2faa234e4e385b82c11337930631d2a1 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2/1903627febdd4806832679268b98ef95.2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:50,130 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2/7414dd4b803f45fca86c7fa465350149.2faa234e4e385b82c11337930631d2a1 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2/7414dd4b803f45fca86c7fa465350149.2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:50,136 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-05 15:53:50,138 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:53:50,138 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1. 2023-05-05 15:53:50,138 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e24ba92d7211d01fc6da0d90d6b8b4e1: 2023-05-05 15:53:50,140 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed e24ba92d7211d01fc6da0d90d6b8b4e1 2023-05-05 15:53:50,141 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:50,141 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9, disabling compactions & flushes 2023-05-05 15:53:50,141 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9. 2023-05-05 15:53:50,141 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9. 2023-05-05 15:53:50,141 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9. after waiting 0 ms 2023-05-05 15:53:50,141 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9. 2023-05-05 15:53:50,141 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=e24ba92d7211d01fc6da0d90d6b8b4e1, regionState=CLOSED 2023-05-05 15:53:50,142 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683302030141"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302030141"}]},"ts":"1683302030141"} 2023-05-05 15:53:50,143 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/d69a911d9c294c03b7032ffbd281155f.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/d69a911d9c294c03b7032ffbd281155f-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/043c3df5b6d24e3cbf5e78974ea62314.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/043c3df5b6d24e3cbf5e78974ea62314-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/17846580efc44fc9b1bbdfb8e6f5a0de.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/17846580efc44fc9b1bbdfb8e6f5a0de-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/8dab1931c3534993ba0f8cd43acfa5f2.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/8dab1931c3534993ba0f8cd43acfa5f2-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-2f51cb98d16e4678ba9160353e4141ce, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-2089f9f8e89a452ab615acf8f64ee9a7, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-523fa9e6142b468d94edaeb8d9b6aeae] to archive 2023-05-05 15:53:50,144 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-05 15:53:50,147 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-05-05 15:53:50,147 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure e24ba92d7211d01fc6da0d90d6b8b4e1, server=jenkins-hbase4.apache.org,42799,1683302022977 in 232 msec 2023-05-05 15:53:50,147 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/d69a911d9c294c03b7032ffbd281155f.2faa234e4e385b82c11337930631d2a1 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/d69a911d9c294c03b7032ffbd281155f.2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:50,149 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e24ba92d7211d01fc6da0d90d6b8b4e1, UNASSIGN in 247 msec 2023-05-05 15:53:50,150 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/043c3df5b6d24e3cbf5e78974ea62314.2faa234e4e385b82c11337930631d2a1 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/043c3df5b6d24e3cbf5e78974ea62314.2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:50,152 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/17846580efc44fc9b1bbdfb8e6f5a0de.2faa234e4e385b82c11337930631d2a1 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/17846580efc44fc9b1bbdfb8e6f5a0de.2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:50,154 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/8dab1931c3534993ba0f8cd43acfa5f2.2faa234e4e385b82c11337930631d2a1 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/8dab1931c3534993ba0f8cd43acfa5f2.2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:50,155 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-2f51cb98d16e4678ba9160353e4141ce to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-2f51cb98d16e4678ba9160353e4141ce 2023-05-05 15:53:50,157 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-2089f9f8e89a452ab615acf8f64ee9a7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-2089f9f8e89a452ab615acf8f64ee9a7 2023-05-05 15:53:50,159 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-523fa9e6142b468d94edaeb8d9b6aeae to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-523fa9e6142b468d94edaeb8d9b6aeae 2023-05-05 15:53:50,161 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/33cfd2a14784445ba210b3c709d236ec.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/33cfd2a14784445ba210b3c709d236ec-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/4991bfadfb32442e904704119d917f44.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/4991bfadfb32442e904704119d917f44-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/1903627febdd4806832679268b98ef95.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/1903627febdd4806832679268b98ef95-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/7414dd4b803f45fca86c7fa465350149.2faa234e4e385b82c11337930631d2a1->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/7414dd4b803f45fca86c7fa465350149-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-bc69b5ee5e594dbd8f6fcdf67defe662, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-863ba511c93d45cfaeb4f9a4e1720d66, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-8d3b8f1928d8481997156b461034e9fa] to archive 2023-05-05 15:53:50,162 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-05 15:53:50,164 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/33cfd2a14784445ba210b3c709d236ec.2faa234e4e385b82c11337930631d2a1 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/33cfd2a14784445ba210b3c709d236ec.2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:50,166 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/4991bfadfb32442e904704119d917f44.2faa234e4e385b82c11337930631d2a1 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/4991bfadfb32442e904704119d917f44.2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:50,168 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/1903627febdd4806832679268b98ef95.2faa234e4e385b82c11337930631d2a1 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/1903627febdd4806832679268b98ef95.2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:50,169 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/7414dd4b803f45fca86c7fa465350149.2faa234e4e385b82c11337930631d2a1 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/7414dd4b803f45fca86c7fa465350149.2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:50,170 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-bc69b5ee5e594dbd8f6fcdf67defe662 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-bc69b5ee5e594dbd8f6fcdf67defe662 2023-05-05 15:53:50,172 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-863ba511c93d45cfaeb4f9a4e1720d66 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-863ba511c93d45cfaeb4f9a4e1720d66 2023-05-05 15:53:50,173 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-8d3b8f1928d8481997156b461034e9fa to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/testSplitTableRegionUnevenDaughter=2faa234e4e385b82c11337930631d2a1-8d3b8f1928d8481997156b461034e9fa 2023-05-05 15:53:50,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-05 15:53:50,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:53:50,182 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9. 2023-05-05 15:53:50,182 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9: 2023-05-05 15:53:50,184 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:50,185 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9, regionState=CLOSED 2023-05-05 15:53:50,185 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683302030185"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302030185"}]},"ts":"1683302030185"} 2023-05-05 15:53:50,190 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-05-05 15:53:50,190 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9, server=jenkins-hbase4.apache.org,42799,1683302022977 in 282 msec 2023-05-05 15:53:50,192 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=19 2023-05-05 15:53:50,192 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9, UNASSIGN in 290 msec 2023-05-05 15:53:50,195 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302030195"}]},"ts":"1683302030195"} 2023-05-05 15:53:50,197 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-05-05 15:53:50,199 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-05-05 15:53:50,202 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 316 msec 2023-05-05 15:53:50,210 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-05-05 15:53:50,211 INFO [Listener at localhost/35117] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-05-05 15:53:50,216 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionUnevenDaughter 2023-05-05 15:53:50,225 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:50,227 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:50,229 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:50,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-05-05 15:53:50,238 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:50,238 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1 2023-05-05 15:53:50,238 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:50,243 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/recovered.edits] 2023-05-05 15:53:50,243 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/recovered.edits] 2023-05-05 15:53:50,243 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/recovered.edits] 2023-05-05 15:53:50,248 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1/09271e080258433fb610e3de0a026492 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf1/09271e080258433fb610e3de0a026492 2023-05-05 15:53:50,249 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/043c3df5b6d24e3cbf5e78974ea62314 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/043c3df5b6d24e3cbf5e78974ea62314 2023-05-05 15:53:50,250 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/132020930e2e449ba169b11bcb82115d to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf1/132020930e2e449ba169b11bcb82115d 2023-05-05 15:53:50,251 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/17846580efc44fc9b1bbdfb8e6f5a0de to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/17846580efc44fc9b1bbdfb8e6f5a0de 2023-05-05 15:53:50,253 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-05 15:53:50,253 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/2089f9f8e89a452ab615acf8f64ee9a7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/2089f9f8e89a452ab615acf8f64ee9a7 2023-05-05 15:53:50,255 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2/1097bb8b308b40dd8e397344d9dbef44 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/cf2/1097bb8b308b40dd8e397344d9dbef44 2023-05-05 15:53:50,256 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/2f51cb98d16e4678ba9160353e4141ce to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/2f51cb98d16e4678ba9160353e4141ce 2023-05-05 15:53:50,256 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/9a36f6eede8a43fd813495dc3632612c to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/cf2/9a36f6eede8a43fd813495dc3632612c 2023-05-05 15:53:50,258 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/523fa9e6142b468d94edaeb8d9b6aeae to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/523fa9e6142b468d94edaeb8d9b6aeae 2023-05-05 15:53:50,267 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/recovered.edits/90.seqid to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1/recovered.edits/90.seqid 2023-05-05 15:53:50,267 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/8dab1931c3534993ba0f8cd43acfa5f2 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/8dab1931c3534993ba0f8cd43acfa5f2 2023-05-05 15:53:50,268 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/e24ba92d7211d01fc6da0d90d6b8b4e1 2023-05-05 15:53:50,270 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/recovered.edits/90.seqid to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9/recovered.edits/90.seqid 2023-05-05 15:53:50,270 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:50,271 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/d69a911d9c294c03b7032ffbd281155f to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf1/d69a911d9c294c03b7032ffbd281155f 2023-05-05 15:53:50,275 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/1903627febdd4806832679268b98ef95 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/1903627febdd4806832679268b98ef95 2023-05-05 15:53:50,277 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/33cfd2a14784445ba210b3c709d236ec to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/33cfd2a14784445ba210b3c709d236ec 2023-05-05 15:53:50,279 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/4991bfadfb32442e904704119d917f44 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/4991bfadfb32442e904704119d917f44 2023-05-05 15:53:50,281 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/7414dd4b803f45fca86c7fa465350149 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/7414dd4b803f45fca86c7fa465350149 2023-05-05 15:53:50,283 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/863ba511c93d45cfaeb4f9a4e1720d66 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/863ba511c93d45cfaeb4f9a4e1720d66 2023-05-05 15:53:50,286 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/8d3b8f1928d8481997156b461034e9fa to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/8d3b8f1928d8481997156b461034e9fa 2023-05-05 15:53:50,290 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/bc69b5ee5e594dbd8f6fcdf67defe662 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/cf2/bc69b5ee5e594dbd8f6fcdf67defe662 2023-05-05 15:53:50,294 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/recovered.edits/85.seqid to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1/recovered.edits/85.seqid 2023-05-05 15:53:50,296 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionUnevenDaughter/2faa234e4e385b82c11337930631d2a1 2023-05-05 15:53:50,296 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-05-05 15:53:50,301 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:50,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-05-05 15:53:50,334 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-05-05 15:53:50,339 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-05-05 15:53:50,340 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-05-05 15:53:50,384 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-05-05 15:53:50,386 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:50,386 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-05-05 15:53:50,386 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302030386"}]},"ts":"9223372036854775807"} 2023-05-05 15:53:50,386 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302030386"}]},"ts":"9223372036854775807"} 2023-05-05 15:53:50,386 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302030386"}]},"ts":"9223372036854775807"} 2023-05-05 15:53:50,390 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-05 15:53:50,390 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 2faa234e4e385b82c11337930631d2a1, NAME => 'testSplitTableRegionUnevenDaughter,,1683302025760.2faa234e4e385b82c11337930631d2a1.', STARTKEY => '', ENDKEY => ''}, {ENCODED => e24ba92d7211d01fc6da0d90d6b8b4e1, NAME => 'testSplitTableRegionUnevenDaughter,,1683302027922.e24ba92d7211d01fc6da0d90d6b8b4e1.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9, NAME => 'testSplitTableRegionUnevenDaughter,26,1683302027922.7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9.', STARTKEY => '26', ENDKEY => ''}] 2023-05-05 15:53:50,390 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-05-05 15:53:50,390 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683302030390"}]},"ts":"9223372036854775807"} 2023-05-05 15:53:50,392 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-05-05 15:53:50,396 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:50,398 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 179 msec 2023-05-05 15:53:50,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-05-05 15:53:50,535 INFO [Listener at localhost/35117] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-05-05 15:53:50,553 INFO [Listener at localhost/35117] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=394 (was 359) Potentially hanging thread: hconnection-0x6277894c-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6277894c-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x603c4d85-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6277894c-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1368172992_15 at /127.0.0.1:36102 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1368172992_15 at /127.0.0.1:35098 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1368172992_15 at /127.0.0.1:54668 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1368172992_15 at /127.0.0.1:36104 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1368172992_15 at /127.0.0.1:54532 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1368172992_15 at /127.0.0.1:36106 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1368172992_15 at /127.0.0.1:36028 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6277894c-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1368172992_15 at /127.0.0.1:54712 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1368172992_15 at /127.0.0.1:36068 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x603c4d85-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1368172992_15 at /127.0.0.1:54918 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1368172992_15 at /127.0.0.1:36086 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6277894c-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase4:42799-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x603c4d85-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1368172992_15 at /127.0.0.1:36130 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1368172992_15 at /127.0.0.1:36076 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6277894c-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x603c4d85-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1368172992_15 at /127.0.0.1:36056 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1368172992_15 at /127.0.0.1:39988 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1368172992_15 at /127.0.0.1:35016 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=864 (was 695) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=347 (was 369), ProcessCount=179 (was 179), AvailableMemoryMB=5318 (was 5396) 2023-05-05 15:53:50,568 INFO [Listener at localhost/35117] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=394, OpenFileDescriptor=864, MaxFileDescriptor=60000, SystemLoadAverage=347, ProcessCount=179, AvailableMemoryMB=5317 2023-05-05 15:53:50,568 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-05 15:53:50,574 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-05 15:53:50,575 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-05 15:53:50,577 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:53:50,579 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-05-05 15:53:50,580 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-05 15:53:50,582 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613 2023-05-05 15:53:50,583 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613 empty. 2023-05-05 15:53:50,584 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613 2023-05-05 15:53:50,584 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-05-05 15:53:50,601 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-05-05 15:53:50,602 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7675): creating {ENCODED => 82d51cc6743c70579253652610cbc613, NAME => 'testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp 2023-05-05 15:53:50,612 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:53:50,612 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1604): Closing 82d51cc6743c70579253652610cbc613, disabling compactions & flushes 2023-05-05 15:53:50,612 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. 2023-05-05 15:53:50,612 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. 2023-05-05 15:53:50,612 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. after waiting 0 ms 2023-05-05 15:53:50,612 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. 2023-05-05 15:53:50,612 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. 2023-05-05 15:53:50,612 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1558): Region close journal for 82d51cc6743c70579253652610cbc613: 2023-05-05 15:53:50,615 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-05-05 15:53:50,616 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683302030616"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302030616"}]},"ts":"1683302030616"} 2023-05-05 15:53:50,619 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-05 15:53:50,620 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-05 15:53:50,620 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302030620"}]},"ts":"1683302030620"} 2023-05-05 15:53:50,622 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-05-05 15:53:50,625 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-05 15:53:50,626 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-05 15:53:50,626 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-05 15:53:50,626 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-05 15:53:50,626 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-05 15:53:50,626 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, ASSIGN}] 2023-05-05 15:53:50,628 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, ASSIGN 2023-05-05 15:53:50,629 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40991,1683302023016; forceNewPlan=false, retain=false 2023-05-05 15:53:50,779 INFO [jenkins-hbase4:44213] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-05 15:53:50,781 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=82d51cc6743c70579253652610cbc613, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:50,781 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683302030781"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302030781"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302030781"}]},"ts":"1683302030781"} 2023-05-05 15:53:50,783 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 82d51cc6743c70579253652610cbc613, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:53:50,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. 2023-05-05 15:53:50,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 82d51cc6743c70579253652610cbc613, NAME => 'testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613.', STARTKEY => '', ENDKEY => ''} 2023-05-05 15:53:50,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:53:50,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:50,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:53:50,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:50,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:50,942 INFO [StoreOpener-82d51cc6743c70579253652610cbc613-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:50,943 DEBUG [StoreOpener-82d51cc6743c70579253652610cbc613-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1 2023-05-05 15:53:50,943 DEBUG [StoreOpener-82d51cc6743c70579253652610cbc613-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1 2023-05-05 15:53:50,944 INFO [StoreOpener-82d51cc6743c70579253652610cbc613-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 82d51cc6743c70579253652610cbc613 columnFamilyName cf1 2023-05-05 15:53:50,944 INFO [StoreOpener-82d51cc6743c70579253652610cbc613-1] regionserver.HStore(310): Store=82d51cc6743c70579253652610cbc613/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:53:50,945 INFO [StoreOpener-82d51cc6743c70579253652610cbc613-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:50,946 DEBUG [StoreOpener-82d51cc6743c70579253652610cbc613-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2 2023-05-05 15:53:50,946 DEBUG [StoreOpener-82d51cc6743c70579253652610cbc613-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2 2023-05-05 15:53:50,948 INFO [StoreOpener-82d51cc6743c70579253652610cbc613-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 82d51cc6743c70579253652610cbc613 columnFamilyName cf2 2023-05-05 15:53:50,949 INFO [StoreOpener-82d51cc6743c70579253652610cbc613-1] regionserver.HStore(310): Store=82d51cc6743c70579253652610cbc613/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:53:50,950 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613 2023-05-05 15:53:50,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613 2023-05-05 15:53:50,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-05 15:53:50,954 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:50,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-05 15:53:50,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 82d51cc6743c70579253652610cbc613; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69483219, jitterRate=0.03538064658641815}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-05 15:53:50,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 82d51cc6743c70579253652610cbc613: 2023-05-05 15:53:50,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613., pid=27, masterSystemTime=1683302030935 2023-05-05 15:53:50,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. 2023-05-05 15:53:50,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. 2023-05-05 15:53:50,961 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=82d51cc6743c70579253652610cbc613, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:50,961 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683302030961"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302030961"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302030961"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302030961"}]},"ts":"1683302030961"} 2023-05-05 15:53:50,966 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-05-05 15:53:50,966 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 82d51cc6743c70579253652610cbc613, server=jenkins-hbase4.apache.org,40991,1683302023016 in 180 msec 2023-05-05 15:53:50,968 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-05-05 15:53:50,968 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, ASSIGN in 340 msec 2023-05-05 15:53:50,969 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-05 15:53:50,969 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302030969"}]},"ts":"1683302030969"} 2023-05-05 15:53:50,971 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-05-05 15:53:50,974 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-05-05 15:53:50,975 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 398 msec 2023-05-05 15:53:51,089 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:53:51,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-05-05 15:53:51,091 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-05-05 15:53:51,092 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:51,093 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-05-05 15:53:51,093 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-05-05 15:53:51,093 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,093 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,094 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,094 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,094 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,094 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,095 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,095 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:53:51,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,096 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,096 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,096 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:51,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:51,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,097 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-05 15:53:51,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:51,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,097 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-05 15:53:51,097 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-05 15:53:51,097 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-05 15:53:51,097 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-05 15:53:51,097 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-05 15:53:51,097 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-05 15:53:51,097 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-05 15:53:51,097 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-05 15:53:51,097 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:51,097 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:51,098 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-05 15:53:51,099 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-05 15:53:51,099 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-05 15:53:51,099 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-05 15:53:51,097 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:51,099 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-05 15:53:51,099 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-05 15:53:51,100 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-05 15:53:51,100 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:53:51,100 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:53:51,100 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. 2023-05-05 15:53:51,100 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. started... 2023-05-05 15:53:51,101 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 82d51cc6743c70579253652610cbc613 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-05 15:53:51,103 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,103 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,104 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:51,103 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,104 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,104 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:51,120 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf1/2dc4637c40d6410ab1613fd8199dc5d4 2023-05-05 15:53:51,144 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf2/635b6eae56774f39a70460b7ccb8d1aa 2023-05-05 15:53:51,149 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf1/2dc4637c40d6410ab1613fd8199dc5d4 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/2dc4637c40d6410ab1613fd8199dc5d4 2023-05-05 15:53:51,154 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/2dc4637c40d6410ab1613fd8199dc5d4, entries=2, sequenceid=6, filesize=4.8 K 2023-05-05 15:53:51,155 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf2/635b6eae56774f39a70460b7ccb8d1aa as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/635b6eae56774f39a70460b7ccb8d1aa 2023-05-05 15:53:51,161 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/635b6eae56774f39a70460b7ccb8d1aa, entries=2, sequenceid=6, filesize=4.8 K 2023-05-05 15:53:51,163 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 82d51cc6743c70579253652610cbc613 in 61ms, sequenceid=6, compaction requested=false 2023-05-05 15:53:51,163 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-05-05 15:53:51,163 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 82d51cc6743c70579253652610cbc613: 2023-05-05 15:53:51,164 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. 2023-05-05 15:53:51,164 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:53:51,164 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:53:51,164 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,164 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-05 15:53:51,164 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-05 15:53:51,165 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,166 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:51,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:51,166 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,166 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:51,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:51,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:51,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:51,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:51,168 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-05-05 15:53:51,168 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@33288ea0[Count = 0] remaining members to acquire global barrier 2023-05-05 15:53:51,168 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-05-05 15:53:51,169 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,170 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,170 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,170 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,170 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,170 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-05 15:53:51,170 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-05 15:53:51,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,170 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-05 15:53:51,170 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-05 15:53:51,170 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:53:51,171 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:53:51,171 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:53:51,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,171 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-05 15:53:51,171 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-05 15:53:51,171 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:53:51,173 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,173 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-05 15:53:51,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:51,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:51,173 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:51,173 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-05 15:53:51,174 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-05 15:53:51,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:51,174 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:51,174 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-05 15:53:51,175 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:51,175 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-05 15:53:51,175 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-05 15:53:51,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:51,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:51,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:51,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:51,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:53:51,180 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-05-05 15:53:51,180 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-05-05 15:53:51,180 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:53:51,180 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:53:51,180 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-05-05 15:53:51,180 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:53:51,182 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,182 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:51,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:51,182 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,182 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,182 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:51,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,182 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:51,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,182 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:51,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:51,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:51,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:51,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:51,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:51,183 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:51,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:51,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:51,184 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:51,184 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:51,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:51,186 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:51,186 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,187 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,190 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,190 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,190 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,190 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:51,190 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:51,190 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,190 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,190 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,191 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,191 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,191 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,191 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,191 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,191 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,191 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,191 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,190 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-05-05 15:53:51,191 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:53:51,190 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:51,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,190 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:51,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,192 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-05-05 15:53:51,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:51,192 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:53:51,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:51,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:51,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:51,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:51,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:51,293 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:53:51,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:53:51,312 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:53:51,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-05-05 15:53:51,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-05-05 15:53:51,316 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-05-05 15:53:51,316 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:51,317 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-05-05 15:53:51,317 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-05-05 15:53:51,318 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,318 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,320 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,320 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,320 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,320 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,320 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,320 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:53:51,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,321 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,321 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,321 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:51,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:51,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:51,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,322 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-05 15:53:51,321 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-05 15:53:51,322 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-05 15:53:51,322 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-05 15:53:51,322 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-05 15:53:51,322 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-05 15:53:51,322 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-05 15:53:51,322 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-05 15:53:51,322 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-05 15:53:51,322 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-05 15:53:51,322 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-05 15:53:51,322 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:51,322 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:51,322 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-05 15:53:51,323 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-05 15:53:51,323 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:51,324 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-05 15:53:51,324 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-05 15:53:51,325 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-05 15:53:51,325 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-05 15:53:51,325 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:53:51,325 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:53:51,325 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. 2023-05-05 15:53:51,323 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-05 15:53:51,325 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-05 15:53:51,325 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. started... 2023-05-05 15:53:51,325 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 82d51cc6743c70579253652610cbc613 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:53:51,326 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,327 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,327 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,327 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:51,327 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,327 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:51,344 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf1/56ff2ff5709e493cb80be0ea26615515 2023-05-05 15:53:51,360 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf2/c743bc8803c947088e8f55d598ec90ee 2023-05-05 15:53:51,366 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf1/56ff2ff5709e493cb80be0ea26615515 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/56ff2ff5709e493cb80be0ea26615515 2023-05-05 15:53:51,371 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/56ff2ff5709e493cb80be0ea26615515, entries=10, sequenceid=19, filesize=5.0 K 2023-05-05 15:53:51,372 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf2/c743bc8803c947088e8f55d598ec90ee as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/c743bc8803c947088e8f55d598ec90ee 2023-05-05 15:53:51,377 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/c743bc8803c947088e8f55d598ec90ee, entries=10, sequenceid=19, filesize=5.0 K 2023-05-05 15:53:51,378 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 82d51cc6743c70579253652610cbc613 in 53ms, sequenceid=19, compaction requested=false 2023-05-05 15:53:51,378 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 82d51cc6743c70579253652610cbc613: 2023-05-05 15:53:51,378 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. 2023-05-05 15:53:51,378 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:53:51,378 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:53:51,378 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,379 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-05 15:53:51,379 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-05 15:53:51,380 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,380 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:51,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:51,381 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,381 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:51,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:51,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:51,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:51,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:51,382 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-05-05 15:53:51,382 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2188e8f2[Count = 0] remaining members to acquire global barrier 2023-05-05 15:53:51,382 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-05-05 15:53:51,383 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,384 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,384 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,384 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,384 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-05 15:53:51,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,384 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,384 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:53:51,384 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-05 15:53:51,384 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-05 15:53:51,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,384 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:53:51,384 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-05 15:53:51,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,384 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:53:51,384 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-05 15:53:51,385 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-05 15:53:51,385 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:53:51,387 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-05 15:53:51,387 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:51,387 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-05 15:53:51,393 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-05 15:53:51,393 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,393 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:51,393 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-05 15:53:51,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:51,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:51,393 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-05 15:53:51,393 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:51,394 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-05 15:53:51,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:51,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:51,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:51,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:51,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:51,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:53:51,400 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-05-05 15:53:51,400 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-05-05 15:53:51,400 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:53:51,400 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:53:51,400 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-05-05 15:53:51,400 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:53:51,402 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,402 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,402 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:51,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:51,402 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:51,402 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,402 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:51,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,402 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:51,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:51,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:51,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:51,408 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:51,409 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:51,409 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:51,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:51,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:51,411 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,411 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,412 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,415 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,415 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,415 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,415 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:51,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,415 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:51,415 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,415 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:51,415 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:51,416 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,415 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-05-05 15:53:51,416 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:53:51,415 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:51,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:51,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:51,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:51,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:51,416 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-05-05 15:53:51,416 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:53:51,416 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:51,417 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,417 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,417 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,417 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,417 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,417 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,417 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,517 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:53:51,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:53:51,537 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:53:51,539 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-05-05 15:53:51,539 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-05-05 15:53:51,539 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-05-05 15:53:51,539 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:51,540 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-05-05 15:53:51,540 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-05-05 15:53:51,541 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,541 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,542 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,542 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,542 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,542 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,543 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,543 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:53:51,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,543 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,543 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,543 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:51,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,544 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-05 15:53:51,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:51,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:51,544 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-05 15:53:51,544 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-05 15:53:51,544 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-05 15:53:51,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,544 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-05 15:53:51,544 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:51,544 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-05 15:53:51,545 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-05 15:53:51,544 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-05 15:53:51,545 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-05 15:53:51,545 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-05 15:53:51,545 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:51,546 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-05 15:53:51,546 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-05 15:53:51,546 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:53:51,546 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:53:51,546 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-05 15:53:51,546 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. 2023-05-05 15:53:51,546 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-05 15:53:51,546 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-05 15:53:51,546 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. started... 2023-05-05 15:53:51,547 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-05 15:53:51,547 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:51,547 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-05 15:53:51,547 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 82d51cc6743c70579253652610cbc613 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:53:51,547 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-05 15:53:51,547 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-05 15:53:51,549 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,551 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,551 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:51,551 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,551 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,551 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:51,563 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf1/384cb59920554d27854b7194e9d15fcf 2023-05-05 15:53:51,578 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf2/8b37c8ad67154ac78eda03e1bd7a7efe 2023-05-05 15:53:51,583 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf1/384cb59920554d27854b7194e9d15fcf as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/384cb59920554d27854b7194e9d15fcf 2023-05-05 15:53:51,589 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/384cb59920554d27854b7194e9d15fcf, entries=10, sequenceid=32, filesize=5.0 K 2023-05-05 15:53:51,590 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf2/8b37c8ad67154ac78eda03e1bd7a7efe as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8b37c8ad67154ac78eda03e1bd7a7efe 2023-05-05 15:53:51,595 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8b37c8ad67154ac78eda03e1bd7a7efe, entries=10, sequenceid=32, filesize=5.0 K 2023-05-05 15:53:51,596 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 82d51cc6743c70579253652610cbc613 in 49ms, sequenceid=32, compaction requested=true 2023-05-05 15:53:51,597 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 82d51cc6743c70579253652610cbc613: 2023-05-05 15:53:51,597 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. 2023-05-05 15:53:51,597 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:53:51,597 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:53:51,597 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,597 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-05 15:53:51,597 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-05 15:53:51,599 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,599 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:51,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:51,599 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,599 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:51,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:51,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:51,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:51,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:51,601 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-05-05 15:53:51,601 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4c04a082[Count = 0] remaining members to acquire global barrier 2023-05-05 15:53:51,601 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-05-05 15:53:51,601 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,603 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,603 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,603 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,603 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-05 15:53:51,603 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-05 15:53:51,603 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,603 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:53:51,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,603 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:53:51,603 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-05 15:53:51,603 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-05 15:53:51,603 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:53:51,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,604 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-05 15:53:51,604 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-05 15:53:51,604 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:53:51,606 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-05 15:53:51,606 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:51,606 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-05 15:53:51,607 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,607 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-05 15:53:51,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:51,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:51,607 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:51,607 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-05 15:53:51,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:51,608 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-05 15:53:51,608 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:51,608 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-05 15:53:51,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:51,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:51,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:51,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:51,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:53:51,611 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-05-05 15:53:51,611 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-05-05 15:53:51,611 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:53:51,611 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:53:51,611 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-05-05 15:53:51,611 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:53:51,613 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,613 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,613 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,613 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,613 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:51,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,613 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:51,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:51,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:51,613 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:51,613 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:51,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:51,614 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:51,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:51,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:51,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:51,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:51,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,614 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:51,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:51,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:51,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,619 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,620 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:51,621 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:51,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,630 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,630 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,630 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:51,630 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,630 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,631 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:51,631 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:51,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,630 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,631 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,631 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,631 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,631 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,631 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,631 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,631 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,631 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,631 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,631 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-05-05 15:53:51,632 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:53:51,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:51,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:51,632 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-05-05 15:53:51,632 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:53:51,631 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:51,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:51,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:51,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:51,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:51,732 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:53:51,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:53:51,754 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:53:51,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-05-05 15:53:51,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-05-05 15:53:51,756 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-05-05 15:53:51,756 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:51,757 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-05-05 15:53:51,757 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-05-05 15:53:51,757 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,757 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,759 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,759 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,759 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,759 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,759 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,759 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:53:51,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,760 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,760 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:51,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:51,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,761 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-05 15:53:51,761 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-05 15:53:51,761 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-05 15:53:51,761 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-05 15:53:51,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,761 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-05 15:53:51,761 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:51,762 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-05 15:53:51,762 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:53:51,762 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-05 15:53:51,762 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. 2023-05-05 15:53:51,762 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. started... 2023-05-05 15:53:51,762 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:53:51,762 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,762 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-05 15:53:51,763 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 82d51cc6743c70579253652610cbc613 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:53:51,763 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-05 15:53:51,763 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:51,764 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-05 15:53:51,764 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-05 15:53:51,764 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-05 15:53:51,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:51,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,764 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-05 15:53:51,765 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-05 15:53:51,765 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-05 15:53:51,765 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-05 15:53:51,765 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:51,767 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-05 15:53:51,767 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-05 15:53:51,767 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-05 15:53:51,768 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,769 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,769 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:51,770 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,772 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,772 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:51,778 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf1/eb967395d76242878a77a4ad94aed787 2023-05-05 15:53:51,795 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf2/8d2decae984d45f3b4dd55c89443967f 2023-05-05 15:53:51,801 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf1/eb967395d76242878a77a4ad94aed787 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/eb967395d76242878a77a4ad94aed787 2023-05-05 15:53:51,807 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/eb967395d76242878a77a4ad94aed787, entries=10, sequenceid=45, filesize=5.0 K 2023-05-05 15:53:51,808 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf2/8d2decae984d45f3b4dd55c89443967f as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8d2decae984d45f3b4dd55c89443967f 2023-05-05 15:53:51,813 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8d2decae984d45f3b4dd55c89443967f, entries=10, sequenceid=45, filesize=5.0 K 2023-05-05 15:53:51,814 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 82d51cc6743c70579253652610cbc613 in 51ms, sequenceid=45, compaction requested=true 2023-05-05 15:53:51,814 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 82d51cc6743c70579253652610cbc613: 2023-05-05 15:53:51,814 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. 2023-05-05 15:53:51,814 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:53:51,814 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:53:51,814 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,814 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-05 15:53:51,815 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-05 15:53:51,816 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,816 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:51,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:51,817 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,817 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:51,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:51,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:51,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:51,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:51,820 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-05-05 15:53:51,820 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@22d04ee0[Count = 0] remaining members to acquire global barrier 2023-05-05 15:53:51,820 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-05-05 15:53:51,820 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,821 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,821 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,821 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,821 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,821 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:53:51,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,821 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-05 15:53:51,821 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-05 15:53:51,821 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-05 15:53:51,821 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:53:51,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,821 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-05 15:53:51,822 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-05 15:53:51,822 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:53:51,822 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-05 15:53:51,822 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:53:51,823 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-05 15:53:51,823 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:51,823 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-05 15:53:51,826 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,826 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-05 15:53:51,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:51,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:51,826 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-05 15:53:51,826 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:51,826 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:51,826 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-05 15:53:51,826 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-05 15:53:51,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:51,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:51,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:51,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:51,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:51,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:53:51,829 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-05-05 15:53:51,829 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-05-05 15:53:51,829 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:53:51,829 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:53:51,829 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-05-05 15:53:51,829 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:53:51,831 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,831 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,831 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,831 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:51,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:51,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:51,831 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,831 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:51,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:51,832 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:51,832 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:51,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:51,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:51,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:51,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:51,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:51,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:51,833 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:51,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,833 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:51,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:51,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:51,836 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,836 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,836 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,839 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,839 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,839 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,839 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,839 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,839 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:51,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,839 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:51,839 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,839 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,839 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:51,840 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,839 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:51,839 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-05-05 15:53:51,840 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:53:51,840 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:51,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:51,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:51,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:51,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:51,840 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,840 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-05-05 15:53:51,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:51,841 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:53:51,840 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:51,841 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:51,841 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,841 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,941 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:53:51,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:53:51,946 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-05-05 15:53:51,947 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-05-05 15:53:51,947 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:53:51,948 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-05-05 15:53:51,963 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:53:51,965 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-05-05 15:53:51,965 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-05-05 15:53:51,965 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-05-05 15:53:51,965 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:51,965 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-05-05 15:53:51,965 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-05-05 15:53:51,966 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,966 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,967 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,967 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,967 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:51,967 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:51,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:51,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,968 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,968 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:51,969 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:53:51,969 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:51,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,969 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-05 15:53:51,969 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:51,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:51,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,969 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-05 15:53:51,969 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-05 15:53:51,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:51,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:51,970 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-05 15:53:51,970 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-05 15:53:51,970 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-05 15:53:51,970 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-05 15:53:51,970 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:51,970 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-05 15:53:51,970 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-05 15:53:51,970 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-05 15:53:51,970 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-05 15:53:51,970 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-05 15:53:51,970 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-05 15:53:51,970 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:51,970 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-05 15:53:51,970 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:51,970 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-05 15:53:51,970 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-05 15:53:51,971 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-05 15:53:51,971 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-05 15:53:51,970 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-05 15:53:51,971 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:53:51,971 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:53:51,971 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. 2023-05-05 15:53:51,971 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. started... 2023-05-05 15:53:51,971 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 82d51cc6743c70579253652610cbc613 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:53:51,973 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,974 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,974 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:51,974 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,974 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:51,974 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:51,990 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf1/6f2500f0662b440683ac5ec2e44b19a9 2023-05-05 15:53:52,009 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf2/d24ededcffd9484698b594c561b7ce37 2023-05-05 15:53:52,015 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf1/6f2500f0662b440683ac5ec2e44b19a9 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/6f2500f0662b440683ac5ec2e44b19a9 2023-05-05 15:53:52,021 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/6f2500f0662b440683ac5ec2e44b19a9, entries=10, sequenceid=58, filesize=5.0 K 2023-05-05 15:53:52,022 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf2/d24ededcffd9484698b594c561b7ce37 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/d24ededcffd9484698b594c561b7ce37 2023-05-05 15:53:52,027 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/d24ededcffd9484698b594c561b7ce37, entries=10, sequenceid=58, filesize=5.0 K 2023-05-05 15:53:52,030 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 82d51cc6743c70579253652610cbc613 in 59ms, sequenceid=58, compaction requested=true 2023-05-05 15:53:52,030 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 82d51cc6743c70579253652610cbc613: 2023-05-05 15:53:52,030 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. 2023-05-05 15:53:52,030 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:53:52,030 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:53:52,030 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,030 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-05 15:53:52,030 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-05 15:53:52,032 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,032 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:52,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:52,032 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,032 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:52,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:52,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:52,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:52,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:52,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:52,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:52,035 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-05-05 15:53:52,035 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@557a9994[Count = 0] remaining members to acquire global barrier 2023-05-05 15:53:52,035 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-05-05 15:53:52,035 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,036 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,036 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,036 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,037 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,037 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-05 15:53:52,037 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-05 15:53:52,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,037 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:53:52,037 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-05 15:53:52,037 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-05 15:53:52,037 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:53:52,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,037 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-05 15:53:52,037 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-05 15:53:52,037 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:53:52,037 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:53:52,040 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-05 15:53:52,040 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:52,040 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-05 15:53:52,041 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,041 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-05 15:53:52,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,041 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:52,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:52,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:52,041 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-05 15:53:52,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:52,042 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-05 15:53:52,042 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:52,042 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-05 15:53:52,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:52,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:52,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:52,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:52,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:52,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:52,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:52,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:52,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:53:52,045 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-05-05 15:53:52,045 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-05-05 15:53:52,045 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:53:52,045 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:53:52,045 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-05-05 15:53:52,045 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:53:52,046 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,046 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,046 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,046 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,046 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:52,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,046 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:52,046 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:52,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:52,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:52,047 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:52,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:52,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:52,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:52,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:52,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:52,047 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:52,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:52,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:52,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:52,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,048 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:52,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:52,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:52,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:52,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,049 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:52,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:52,049 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:52,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:52,050 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:52,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:52,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,061 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,061 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:52,061 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:52,061 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:52,061 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:52,061 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:52,061 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:52,061 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:52,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:52,061 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:52,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-05-05 15:53:52,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:53:52,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:52,061 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:52,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:52,061 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:52,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:52,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:52,062 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:52,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:52,062 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:52,062 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-05-05 15:53:52,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:52,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:52,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:52,062 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:52,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:52,062 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:53:52,062 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:52,063 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:52,063 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,063 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,163 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:53:52,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:53:52,181 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:53:52,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-05-05 15:53:52,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-05-05 15:53:52,183 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-05-05 15:53:52,183 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:52,183 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-05-05 15:53:52,183 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-05-05 15:53:52,184 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,184 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:52,185 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,185 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:52,185 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:52,185 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:52,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:52,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:52,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:52,185 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,186 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:53:52,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:52,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:52,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:52,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:52,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:52,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:52,186 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,186 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,186 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:52,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:52,187 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-05 15:53:52,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:52,187 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-05 15:53:52,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:52,187 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-05 15:53:52,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:52,187 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-05 15:53:52,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:52,187 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:52,187 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-05 15:53:52,187 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-05 15:53:52,187 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-05 15:53:52,187 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-05 15:53:52,187 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-05 15:53:52,188 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:53:52,188 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-05 15:53:52,188 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-05 15:53:52,188 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:52,188 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-05 15:53:52,188 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. 2023-05-05 15:53:52,188 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-05 15:53:52,188 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:52,188 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:53:52,188 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-05 15:53:52,188 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-05 15:53:52,188 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. started... 2023-05-05 15:53:52,188 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-05 15:53:52,188 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-05 15:53:52,188 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-05 15:53:52,188 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-05 15:53:52,188 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 82d51cc6743c70579253652610cbc613 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:53:52,190 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,190 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,190 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,190 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:52,190 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,190 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:52,203 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf1/7bc278e48d844abda430f63025011cc7 2023-05-05 15:53:52,222 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf2/952352901c844cc4bcf7b2a33b5c189e 2023-05-05 15:53:52,228 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf1/7bc278e48d844abda430f63025011cc7 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/7bc278e48d844abda430f63025011cc7 2023-05-05 15:53:52,233 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/7bc278e48d844abda430f63025011cc7, entries=10, sequenceid=71, filesize=5.0 K 2023-05-05 15:53:52,234 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf2/952352901c844cc4bcf7b2a33b5c189e as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/952352901c844cc4bcf7b2a33b5c189e 2023-05-05 15:53:52,240 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/952352901c844cc4bcf7b2a33b5c189e, entries=10, sequenceid=71, filesize=5.0 K 2023-05-05 15:53:52,241 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 82d51cc6743c70579253652610cbc613 in 53ms, sequenceid=71, compaction requested=true 2023-05-05 15:53:52,241 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 82d51cc6743c70579253652610cbc613: 2023-05-05 15:53:52,241 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. 2023-05-05 15:53:52,241 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:53:52,241 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:53:52,241 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,242 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-05 15:53:52,242 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-05 15:53:52,243 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,243 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:52,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:52,244 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,244 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:52,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:52,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:52,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:52,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:52,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:52,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:52,245 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-05-05 15:53:52,246 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@335334b1[Count = 0] remaining members to acquire global barrier 2023-05-05 15:53:52,246 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-05-05 15:53:52,246 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,248 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,248 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,248 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,248 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-05 15:53:52,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,248 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,248 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-05 15:53:52,248 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:53:52,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,248 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-05 15:53:52,249 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-05 15:53:52,249 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-05 15:53:52,248 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-05 15:53:52,249 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:53:52,249 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:53:52,249 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:53:52,250 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-05 15:53:52,250 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-05 15:53:52,250 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:52,250 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,250 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-05 15:53:52,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:52,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:52,250 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:52,250 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-05 15:53:52,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:52,251 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-05 15:53:52,251 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:52,251 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-05 15:53:52,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:52,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:52,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:52,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:52,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:52,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:52,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:52,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:52,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:53:52,254 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-05-05 15:53:52,254 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-05-05 15:53:52,254 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:53:52,254 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:53:52,254 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-05-05 15:53:52,254 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:53:52,255 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,255 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,255 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:52,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:52,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,255 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:52,255 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,255 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:52,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:52,256 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:52,255 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:52,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:52,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:52,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:52,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:52,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:52,256 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:52,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:52,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:52,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:52,257 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:52,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:52,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:52,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:52,258 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:52,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-05 15:53:52,259 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:52,259 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:52,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:52,263 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,263 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:52,263 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:52,263 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:52,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:52,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:52,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:52,263 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:52,263 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:52,263 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-05-05 15:53:52,263 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:53:52,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:52,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:52,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:52,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:52,263 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:52,263 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:52,263 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:52,264 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-05-05 15:53:52,264 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:53:52,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:52,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:52,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:52,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:52,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:52,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-05 15:53:52,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:52,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:52,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-05 15:53:52,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-05 15:53:52,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:52,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:52,364 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:53:52,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:53:52,627 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-05-05 15:53:52,628 DEBUG [Listener at localhost/35117] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:52,630 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:52,631 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:52,631 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:52,631 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:52,632 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:53:52,632 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:52,632 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:53:52,632 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:53:52,880 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:52,881 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:53:52,881 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:53:52,882 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:53:52,884 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-05-05 15:53:52,884 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:53:52,885 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:53:52,885 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:53:52,885 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:53:52,885 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:53:52,888 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-05-05 15:53:52,889 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:53:52,890 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:53:52,890 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:53:52,890 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:53:52,891 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:52,891 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:52,891 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:52,892 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:52,892 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:53:52,893 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:52,893 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:53:52,893 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:52,893 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:52,893 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:52,893 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-05 15:53:52,894 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:53:52,895 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:53:52,896 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302062896 2023-05-05 15:53:52,897 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:53:52,897 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:52,901 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:53:52,901 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=82d51cc6743c70579253652610cbc613, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40991,1683302023016, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016, openSeqNum=2 2023-05-05 15:53:52,901 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:53:52,901 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302092901 2023-05-05 15:53:52,901 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302152901 2023-05-05 15:53:52,901 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-05 15:53:52,902 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:53:52,904 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:53:52,904 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:52,904 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:53:52,904 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:53:53,152 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:53,152 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:53:53,152 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:53:53,152 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:53:53,152 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:53:53,152 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:53:53,152 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:53:53,153 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:53:53,153 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:53:53,153 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:53:53,155 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-05-05 15:53:53,155 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:53:53,156 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:53:53,156 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:53:53,156 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:53:53,156 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:53,157 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:53,157 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:53,157 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:53,158 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:53:53,158 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:53,158 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:53:53,158 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:53,159 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:53,159 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:53,159 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-05 15:53:53,159 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:53:53,160 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:53:53,160 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302063160 2023-05-05 15:53:53,162 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:53,162 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:53:53,164 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:53:53,164 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=82d51cc6743c70579253652610cbc613, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40991,1683302023016, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016, openSeqNum=2 2023-05-05 15:53:53,165 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:53:53,165 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302093165 2023-05-05 15:53:53,165 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302153165 2023-05-05 15:53:53,165 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-05 15:53:53,165 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:53:53,167 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN}] 2023-05-05 15:53:53,167 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:53:53,167 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:53,167 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:53:53,167 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:53:53,415 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:53,415 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:53:53,415 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:53:53,415 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:53:53,416 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:53:53,416 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:53:53,417 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:53:53,417 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:53:53,417 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:53:53,417 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:53:53,419 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-05-05 15:53:53,419 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:53:53,420 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:53:53,420 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:53:53,420 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:53:53,421 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:53,421 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:53,421 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:53,421 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:53,422 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:53:53,422 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:53,422 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:53:53,422 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:53,422 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:53,422 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:53,423 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-05 15:53:53,423 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:53:53,424 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:53:53,424 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302063424 2023-05-05 15:53:53,425 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:53:53,425 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:53,428 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:53:53,428 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=82d51cc6743c70579253652610cbc613, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40991,1683302023016, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016, openSeqNum=2 2023-05-05 15:53:53,428 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:53:53,428 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302093428 2023-05-05 15:53:53,428 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302153428 2023-05-05 15:53:53,429 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-05 15:53:53,429 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN}] 2023-05-05 15:53:53,429 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:53:53,431 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN 2023-05-05 15:53:53,432 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=82d51cc6743c70579253652610cbc613, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:53,432 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683302033432"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302033432"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302033432"}]},"ts":"1683302033432"} 2023-05-05 15:53:53,434 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 82d51cc6743c70579253652610cbc613, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:53:53,434 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:53:53,434 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN 2023-05-05 15:53:53,434 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:53:53,434 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:53:53,679 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:53,679 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:53:53,679 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:53:53,679 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:53:53,679 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:53:53,679 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:53:53,679 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:53:53,679 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:53:53,680 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:53:53,680 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:53:53,682 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-05 15:53:53,682 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-05-05 15:53:53,682 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:53:53,682 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:53:53,683 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:53:53,683 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:53:53,683 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:53,684 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:53,684 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:53,684 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:53,684 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:53:53,685 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:53,685 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN 2023-05-05 15:53:53,686 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:53:53,686 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:53,686 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:53,686 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:53,686 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:53:53,686 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN 2023-05-05 15:53:53,686 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-05 15:53:53,687 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:53:53,687 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:53:53,687 INFO [Listener at localhost/35117] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613 to restore RIT 2023-05-05 15:53:53,688 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302063688 2023-05-05 15:53:53,689 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:53:53,689 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN 2023-05-05 15:53:53,691 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:53:53,691 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=82d51cc6743c70579253652610cbc613, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,40991,1683302023016, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016, openSeqNum=2 2023-05-05 15:53:53,691 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:53:53,691 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302093691 2023-05-05 15:53:53,691 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302153691 2023-05-05 15:53:53,691 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-05 15:53:53,692 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=82d51cc6743c70579253652610cbc613, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:53,692 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683302033692"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302033692"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302033692"}]},"ts":"1683302033692"} 2023-05-05 15:53:53,694 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 82d51cc6743c70579253652610cbc613, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:53:53,694 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:53:53,847 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:53,848 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 82d51cc6743c70579253652610cbc613, disabling compactions & flushes 2023-05-05 15:53:53,848 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. 2023-05-05 15:53:53,848 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. 2023-05-05 15:53:53,848 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. after waiting 0 ms 2023-05-05 15:53:53,848 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. 2023-05-05 15:53:53,848 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 82d51cc6743c70579253652610cbc613 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-05 15:53:53,859 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf1/1a5bf8c567904f709a27885b51fb83bb 2023-05-05 15:53:53,873 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf2/1d51d0031db74e5fb0ec38addeb97226 2023-05-05 15:53:53,879 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf1/1a5bf8c567904f709a27885b51fb83bb as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/1a5bf8c567904f709a27885b51fb83bb 2023-05-05 15:53:53,884 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/1a5bf8c567904f709a27885b51fb83bb, entries=8, sequenceid=82, filesize=5.0 K 2023-05-05 15:53:53,885 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/.tmp/cf2/1d51d0031db74e5fb0ec38addeb97226 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/1d51d0031db74e5fb0ec38addeb97226 2023-05-05 15:53:53,890 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/1d51d0031db74e5fb0ec38addeb97226, entries=8, sequenceid=82, filesize=5.0 K 2023-05-05 15:53:53,891 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 82d51cc6743c70579253652610cbc613 in 43ms, sequenceid=82, compaction requested=true 2023-05-05 15:53:53,914 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-05 15:53:53,915 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:53:53,915 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. 2023-05-05 15:53:53,915 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 82d51cc6743c70579253652610cbc613: 2023-05-05 15:53:53,917 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:53,918 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=82d51cc6743c70579253652610cbc613, regionState=CLOSED 2023-05-05 15:53:53,918 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683302033918"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302033918"}]},"ts":"1683302033918"} 2023-05-05 15:53:53,920 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:53:53,920 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 82d51cc6743c70579253652610cbc613, server=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:53,920 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:53:53,920 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 82d51cc6743c70579253652610cbc613, server=jenkins-hbase4.apache.org,40991,1683302023016 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:53:53,942 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:53,942 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:53:53,942 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:53:53,942 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:53:53,943 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:53:53,943 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:53:53,943 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:53:53,943 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:53:53,943 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:53:53,944 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:53:53,946 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-05 15:53:53,946 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-05-05 15:53:53,946 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-05-05 15:53:53,946 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:53:53,947 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:53:53,947 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:53:53,947 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:53:53,948 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:53,948 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:53,948 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:53,949 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:53,949 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:53:53,950 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:53,950 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN 2023-05-05 15:53:53,950 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 82d51cc6743c70579253652610cbc613, server=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:53,950 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:53:53,951 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:53,951 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:53,951 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:53,951 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:53:53,951 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN 2023-05-05 15:53:53,951 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 82d51cc6743c70579253652610cbc613, server=jenkins-hbase4.apache.org,40991,1683302023016 held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:53:53,951 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-05 15:53:53,952 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:53:53,952 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:53:53,953 INFO [Listener at localhost/35117] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613 to restore RIT 2023-05-05 15:53:53,953 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302063953 2023-05-05 15:53:53,954 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 82d51cc6743c70579253652610cbc613, server=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:53,954 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:53:53,956 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:53:53,956 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=82d51cc6743c70579253652610cbc613, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40991,1683302023016, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016, openSeqNum=2 2023-05-05 15:53:53,957 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:53:53,957 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302093957 2023-05-05 15:53:53,957 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302153957 2023-05-05 15:53:53,957 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-05 15:53:53,957 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=82d51cc6743c70579253652610cbc613, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:53,957 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683302033957"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302033957"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302033957"}]},"ts":"1683302033957"} 2023-05-05 15:53:53,959 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:53:53,962 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-05-05 15:53:53,962 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 82d51cc6743c70579253652610cbc613, server=jenkins-hbase4.apache.org,40991,1683302023016 in 266 msec 2023-05-05 15:53:53,963 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:53:53,963 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN 2023-05-05 15:53:53,963 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:53:53,963 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:53:54,207 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:54,207 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:53:54,207 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:53:54,207 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:53:54,207 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:53:54,208 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:53:54,208 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:53:54,208 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:53:54,208 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:53:54,209 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:53:54,210 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-05 15:53:54,210 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-05-05 15:53:54,210 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-05 15:53:54,210 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:53:54,211 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:53:54,211 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:53:54,211 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:53:54,212 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:54,212 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:54,212 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:54,212 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:54,213 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:53:54,213 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:54,214 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN 2023-05-05 15:53:54,214 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 82d51cc6743c70579253652610cbc613, server=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:54,214 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:53:54,214 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:54,214 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:54,214 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:54,214 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:53:54,214 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 82d51cc6743c70579253652610cbc613, server=jenkins-hbase4.apache.org,40991,1683302023016 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:53:54,214 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-05 15:53:54,215 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:53:54,215 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:53:54,216 INFO [Listener at localhost/35117] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613 to restore RIT 2023-05-05 15:53:54,216 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302064216 2023-05-05 15:53:54,217 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:53:54,217 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN 2023-05-05 15:53:54,219 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:53:54,220 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=82d51cc6743c70579253652610cbc613, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40991,1683302023016, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016, openSeqNum=2 2023-05-05 15:53:54,220 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:53:54,220 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302094220 2023-05-05 15:53:54,220 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302154220 2023-05-05 15:53:54,220 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-05 15:53:54,220 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN 2023-05-05 15:53:54,222 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:53:54,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-05-05 15:53:54,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN in 793 msec 2023-05-05 15:53:54,228 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:53:54,228 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:54,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:53:54,228 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:53:54,470 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:54,470 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:53:54,471 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:53:54,471 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:53:54,472 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:53:54,472 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:53:54,472 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:53:54,472 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:53:54,472 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:53:54,473 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:53:54,474 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-05 15:53:54,474 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-05 15:53:54,474 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-05 15:53:54,475 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:53:54,475 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:53:54,475 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:53:54,475 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:53:54,476 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:54,476 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:54,476 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:54,477 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:54,477 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:53:54,477 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:54,478 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN 2023-05-05 15:53:54,478 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 82d51cc6743c70579253652610cbc613, server=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:54,478 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:53:54,478 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:53:54,478 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 82d51cc6743c70579253652610cbc613, server=jenkins-hbase4.apache.org,40991,1683302023016 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:53:54,478 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-05 15:53:54,478 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:53:54,479 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:53:54,479 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302064479 2023-05-05 15:53:54,481 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:53:54,481 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:54,483 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:53:54,483 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=82d51cc6743c70579253652610cbc613, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40991,1683302023016, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016, openSeqNum=2 2023-05-05 15:53:54,483 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:53:54,483 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302094483 2023-05-05 15:53:54,484 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302154484 2023-05-05 15:53:54,484 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-05 15:53:54,484 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:54,484 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:54,484 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:54,488 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:53:54,499 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=82d51cc6743c70579253652610cbc613, threads=14 2023-05-05 15:53:54,500 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/1d51d0031db74e5fb0ec38addeb97226 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,500 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/635b6eae56774f39a70460b7ccb8d1aa for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,500 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8b37c8ad67154ac78eda03e1bd7a7efe for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,500 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/952352901c844cc4bcf7b2a33b5c189e for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,500 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8d2decae984d45f3b4dd55c89443967f for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,504 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/c743bc8803c947088e8f55d598ec90ee for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,504 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/d24ededcffd9484698b594c561b7ce37 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,504 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/1a5bf8c567904f709a27885b51fb83bb for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,505 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/2dc4637c40d6410ab1613fd8199dc5d4 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,505 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/384cb59920554d27854b7194e9d15fcf for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,505 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/56ff2ff5709e493cb80be0ea26615515 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,508 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/6f2500f0662b440683ac5ec2e44b19a9 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,508 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/7bc278e48d844abda430f63025011cc7 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,508 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/eb967395d76242878a77a4ad94aed787 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,696 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/eb967395d76242878a77a4ad94aed787 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,698 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/6f2500f0662b440683ac5ec2e44b19a9 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,698 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/952352901c844cc4bcf7b2a33b5c189e for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,714 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/1d51d0031db74e5fb0ec38addeb97226 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,715 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/c743bc8803c947088e8f55d598ec90ee for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,715 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/384cb59920554d27854b7194e9d15fcf for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,728 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8b37c8ad67154ac78eda03e1bd7a7efe for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,728 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/635b6eae56774f39a70460b7ccb8d1aa for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,735 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/d24ededcffd9484698b594c561b7ce37 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,737 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/7bc278e48d844abda430f63025011cc7 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,744 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/2dc4637c40d6410ab1613fd8199dc5d4 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,744 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/56ff2ff5709e493cb80be0ea26615515 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,748 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8d2decae984d45f3b4dd55c89443967f for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,750 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/1a5bf8c567904f709a27885b51fb83bb for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:54,751 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 82d51cc6743c70579253652610cbc613 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-05 15:53:54,788 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:53:54,788 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:54,788 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:53:54,788 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:53:54,984 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:54,984 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:53:54,984 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:53:54,984 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:53:54,985 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:53:54,986 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:53:54,986 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:53:54,986 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:53:54,986 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:53:54,987 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:53:54,988 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-05-05 15:53:54,988 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-05 15:53:54,989 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-05 15:53:54,989 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:53:54,990 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:53:54,990 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:53:54,991 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:53:54,991 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:54,992 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:54,992 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:54,993 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:54,993 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:53:54,994 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:54,994 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN 2023-05-05 15:53:54,994 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 82d51cc6743c70579253652610cbc613, server=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:54,995 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:53:54,995 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:54,995 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:54,995 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:54,995 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:53:54,995 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 82d51cc6743c70579253652610cbc613, server=jenkins-hbase4.apache.org,40991,1683302023016 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:53:54,995 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-05-05 15:53:54,997 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:53:54,997 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:53:54,998 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302064998 2023-05-05 15:53:54,999 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:54,999 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:53:55,002 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:53:55,002 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=82d51cc6743c70579253652610cbc613, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40991,1683302023016, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016, openSeqNum=2 2023-05-05 15:53:55,002 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:53:55,002 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302095002 2023-05-05 15:53:55,002 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302155002 2023-05-05 15:53:55,002 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-05 15:53:55,013 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=82d51cc6743c70579253652610cbc613, threads=14 2023-05-05 15:53:55,016 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/1d51d0031db74e5fb0ec38addeb97226 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,016 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/635b6eae56774f39a70460b7ccb8d1aa for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,016 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8b37c8ad67154ac78eda03e1bd7a7efe for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,016 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8d2decae984d45f3b4dd55c89443967f for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,016 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/952352901c844cc4bcf7b2a33b5c189e for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,017 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/c743bc8803c947088e8f55d598ec90ee for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,017 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/1a5bf8c567904f709a27885b51fb83bb for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,017 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/2dc4637c40d6410ab1613fd8199dc5d4 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,017 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/56ff2ff5709e493cb80be0ea26615515 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,017 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/d24ededcffd9484698b594c561b7ce37 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,017 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/384cb59920554d27854b7194e9d15fcf for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,018 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/7bc278e48d844abda430f63025011cc7 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,018 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/6f2500f0662b440683ac5ec2e44b19a9 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,019 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/eb967395d76242878a77a4ad94aed787 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,269 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/384cb59920554d27854b7194e9d15fcf for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,282 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8d2decae984d45f3b4dd55c89443967f for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,284 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/6f2500f0662b440683ac5ec2e44b19a9 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,284 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8b37c8ad67154ac78eda03e1bd7a7efe for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,285 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/2dc4637c40d6410ab1613fd8199dc5d4 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,308 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/c743bc8803c947088e8f55d598ec90ee for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,313 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/7bc278e48d844abda430f63025011cc7 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,313 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/635b6eae56774f39a70460b7ccb8d1aa for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,314 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/1a5bf8c567904f709a27885b51fb83bb for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,315 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/d24ededcffd9484698b594c561b7ce37 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,316 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/56ff2ff5709e493cb80be0ea26615515 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,317 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/952352901c844cc4bcf7b2a33b5c189e for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,322 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/1d51d0031db74e5fb0ec38addeb97226 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,641 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-05 15:53:55,648 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/eb967395d76242878a77a4ad94aed787 for region: 82d51cc6743c70579253652610cbc613 2023-05-05 15:53:55,648 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 82d51cc6743c70579253652610cbc613 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-05 15:53:55,715 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:53:55,721 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-05 15:53:55,723 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-05 15:53:55,723 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:53:55,723 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:55,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:53:55,723 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:53:55,753 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:55,753 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:53:55,753 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:53:55,753 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:53:55,754 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:53:55,754 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:53:55,755 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:53:55,755 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:53:55,755 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:53:55,756 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:53:55,757 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-05-05 15:53:55,757 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-05 15:53:55,757 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-05 15:53:55,757 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:53:55,758 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:53:55,758 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:53:55,759 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:53:55,759 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:55,760 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:55,760 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:55,760 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:55,761 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:53:55,762 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:55,762 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN 2023-05-05 15:53:55,762 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 82d51cc6743c70579253652610cbc613, server=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:55,762 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:53:55,762 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:55,762 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:55,762 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:55,763 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:53:55,763 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 82d51cc6743c70579253652610cbc613, server=jenkins-hbase4.apache.org,40991,1683302023016 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:53:55,763 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-05 15:53:55,764 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:53:55,764 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:53:55,765 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302065765 2023-05-05 15:53:55,766 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:55,766 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:53:55,770 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:53:55,770 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=82d51cc6743c70579253652610cbc613, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40991,1683302023016, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016, openSeqNum=2 2023-05-05 15:53:55,770 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:53:55,770 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302095770 2023-05-05 15:53:55,770 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302155770 2023-05-05 15:53:55,771 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-05 15:53:55,774 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:53:55,776 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:53:55,776 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:55,776 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:53:55,776 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:53:56,021 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-05 15:53:56,021 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:53:56,021 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:53:56,021 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:53:56,022 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:53:56,022 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:53:56,022 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:53:56,022 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:53:56,022 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:53:56,023 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:53:56,024 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-05-05 15:53:56,024 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-05 15:53:56,024 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-05 15:53:56,025 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:53:56,025 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:53:56,025 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:53:56,025 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:53:56,026 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:56,026 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:53:56,026 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:56,026 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:53:56,027 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:53:56,027 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:56,027 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN 2023-05-05 15:53:56,027 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 82d51cc6743c70579253652610cbc613, server=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:56,027 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:53:56,028 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:56,028 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:56,028 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:56,028 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=82d51cc6743c70579253652610cbc613, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:53:56,028 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 82d51cc6743c70579253652610cbc613, server=jenkins-hbase4.apache.org,40991,1683302023016 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:53:56,028 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-05 15:53:56,029 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:53:56,029 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:53:56,029 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302066029 2023-05-05 15:53:56,031 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:56,031 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:53:56,033 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:53:56,033 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=82d51cc6743c70579253652610cbc613, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40991,1683302023016, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016, openSeqNum=2 2023-05-05 15:53:56,033 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:53:56,033 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302096033 2023-05-05 15:53:56,033 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302156033 2023-05-05 15:53:56,033 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-05 15:53:56,037 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683302036037"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1683302036037"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1683302036037"}]},"ts":"1683302036037"} 2023-05-05 15:53:56,037 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683302036037"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302036037"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302036037"}]},"ts":"1683302036037"} 2023-05-05 15:53:56,037 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683302036037"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302036037"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302036037"}]},"ts":"1683302036037"} 2023-05-05 15:53:56,043 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f0ed874c66cda1037a613cc845c4e7e, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=e800c4983abdf0cef46c4396eec67564, ASSIGN}] 2023-05-05 15:53:56,044 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=e800c4983abdf0cef46c4396eec67564, ASSIGN 2023-05-05 15:53:56,045 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=e800c4983abdf0cef46c4396eec67564, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40991,1683302023016; forceNewPlan=false, retain=false 2023-05-05 15:53:56,045 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f0ed874c66cda1037a613cc845c4e7e, ASSIGN 2023-05-05 15:53:56,046 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f0ed874c66cda1037a613cc845c4e7e, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40991,1683302023016; forceNewPlan=false, retain=false 2023-05-05 15:53:56,195 INFO [jenkins-hbase4:44213] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-05 15:53:56,197 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=6f0ed874c66cda1037a613cc845c4e7e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:56,197 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683302036196"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302036196"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302036196"}]},"ts":"1683302036196"} 2023-05-05 15:53:56,199 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure 6f0ed874c66cda1037a613cc845c4e7e, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:53:56,202 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=e800c4983abdf0cef46c4396eec67564, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:56,203 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683302036202"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302036202"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302036202"}]},"ts":"1683302036202"} 2023-05-05 15:53:56,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure e800c4983abdf0cef46c4396eec67564, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:53:56,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e. 2023-05-05 15:53:56,353 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6f0ed874c66cda1037a613cc845c4e7e, NAME => 'testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.', STARTKEY => '', ENDKEY => '41'} 2023-05-05 15:53:56,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:53:56,354 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 6f0ed874c66cda1037a613cc845c4e7e 2023-05-05 15:53:56,354 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:53:56,354 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6f0ed874c66cda1037a613cc845c4e7e 2023-05-05 15:53:56,354 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6f0ed874c66cda1037a613cc845c4e7e 2023-05-05 15:53:56,355 INFO [StoreOpener-6f0ed874c66cda1037a613cc845c4e7e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6f0ed874c66cda1037a613cc845c4e7e 2023-05-05 15:53:56,357 DEBUG [StoreOpener-6f0ed874c66cda1037a613cc845c4e7e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1 2023-05-05 15:53:56,357 DEBUG [StoreOpener-6f0ed874c66cda1037a613cc845c4e7e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1 2023-05-05 15:53:56,357 INFO [StoreOpener-6f0ed874c66cda1037a613cc845c4e7e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6f0ed874c66cda1037a613cc845c4e7e columnFamilyName cf1 2023-05-05 15:53:56,373 DEBUG [StoreOpener-6f0ed874c66cda1037a613cc845c4e7e-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/1a5bf8c567904f709a27885b51fb83bb.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/1a5bf8c567904f709a27885b51fb83bb-bottom 2023-05-05 15:53:56,378 DEBUG [StoreOpener-6f0ed874c66cda1037a613cc845c4e7e-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/2dc4637c40d6410ab1613fd8199dc5d4.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/2dc4637c40d6410ab1613fd8199dc5d4-bottom 2023-05-05 15:53:56,384 DEBUG [StoreOpener-6f0ed874c66cda1037a613cc845c4e7e-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/384cb59920554d27854b7194e9d15fcf.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/384cb59920554d27854b7194e9d15fcf-bottom 2023-05-05 15:53:56,390 DEBUG [StoreOpener-6f0ed874c66cda1037a613cc845c4e7e-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/56ff2ff5709e493cb80be0ea26615515.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/56ff2ff5709e493cb80be0ea26615515-bottom 2023-05-05 15:53:56,394 DEBUG [StoreOpener-6f0ed874c66cda1037a613cc845c4e7e-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/6f2500f0662b440683ac5ec2e44b19a9.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/6f2500f0662b440683ac5ec2e44b19a9-bottom 2023-05-05 15:53:56,400 DEBUG [StoreOpener-6f0ed874c66cda1037a613cc845c4e7e-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/7bc278e48d844abda430f63025011cc7.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/7bc278e48d844abda430f63025011cc7-bottom 2023-05-05 15:53:56,404 DEBUG [StoreOpener-6f0ed874c66cda1037a613cc845c4e7e-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/eb967395d76242878a77a4ad94aed787.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/eb967395d76242878a77a4ad94aed787-bottom 2023-05-05 15:53:56,404 INFO [StoreOpener-6f0ed874c66cda1037a613cc845c4e7e-1] regionserver.HStore(310): Store=6f0ed874c66cda1037a613cc845c4e7e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:53:56,404 INFO [StoreOpener-6f0ed874c66cda1037a613cc845c4e7e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6f0ed874c66cda1037a613cc845c4e7e 2023-05-05 15:53:56,405 DEBUG [StoreOpener-6f0ed874c66cda1037a613cc845c4e7e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2 2023-05-05 15:53:56,405 DEBUG [StoreOpener-6f0ed874c66cda1037a613cc845c4e7e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2 2023-05-05 15:53:56,406 INFO [StoreOpener-6f0ed874c66cda1037a613cc845c4e7e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6f0ed874c66cda1037a613cc845c4e7e columnFamilyName cf2 2023-05-05 15:53:56,420 DEBUG [StoreOpener-6f0ed874c66cda1037a613cc845c4e7e-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/1d51d0031db74e5fb0ec38addeb97226.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/1d51d0031db74e5fb0ec38addeb97226-bottom 2023-05-05 15:53:56,424 DEBUG [StoreOpener-6f0ed874c66cda1037a613cc845c4e7e-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/635b6eae56774f39a70460b7ccb8d1aa.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/635b6eae56774f39a70460b7ccb8d1aa-bottom 2023-05-05 15:53:56,429 DEBUG [StoreOpener-6f0ed874c66cda1037a613cc845c4e7e-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/8b37c8ad67154ac78eda03e1bd7a7efe.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8b37c8ad67154ac78eda03e1bd7a7efe-bottom 2023-05-05 15:53:56,433 DEBUG [StoreOpener-6f0ed874c66cda1037a613cc845c4e7e-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/8d2decae984d45f3b4dd55c89443967f.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8d2decae984d45f3b4dd55c89443967f-bottom 2023-05-05 15:53:56,438 DEBUG [StoreOpener-6f0ed874c66cda1037a613cc845c4e7e-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/952352901c844cc4bcf7b2a33b5c189e.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/952352901c844cc4bcf7b2a33b5c189e-bottom 2023-05-05 15:53:56,442 DEBUG [StoreOpener-6f0ed874c66cda1037a613cc845c4e7e-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/c743bc8803c947088e8f55d598ec90ee.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/c743bc8803c947088e8f55d598ec90ee-bottom 2023-05-05 15:53:56,450 DEBUG [StoreOpener-6f0ed874c66cda1037a613cc845c4e7e-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/d24ededcffd9484698b594c561b7ce37.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/d24ededcffd9484698b594c561b7ce37-bottom 2023-05-05 15:53:56,450 INFO [StoreOpener-6f0ed874c66cda1037a613cc845c4e7e-1] regionserver.HStore(310): Store=6f0ed874c66cda1037a613cc845c4e7e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:53:56,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e 2023-05-05 15:53:56,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e 2023-05-05 15:53:56,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-05 15:53:56,455 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6f0ed874c66cda1037a613cc845c4e7e 2023-05-05 15:53:56,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6f0ed874c66cda1037a613cc845c4e7e; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68032666, jitterRate=0.013765722513198853}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-05 15:53:56,455 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6f0ed874c66cda1037a613cc845c4e7e: 2023-05-05 15:53:56,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e., pid=33, masterSystemTime=1683302036350 2023-05-05 15:53:56,457 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:53:56,458 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-05 15:53:56,458 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-05 15:53:56,458 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-05 15:53:56,458 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e. 2023-05-05 15:53:56,458 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e. 2023-05-05 15:53:56,458 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1912): 6f0ed874c66cda1037a613cc845c4e7e/cf2 is initiating minor compaction (all files) 2023-05-05 15:53:56,458 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1912): 6f0ed874c66cda1037a613cc845c4e7e/cf1 is initiating minor compaction (all files) 2023-05-05 15:53:56,458 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 6f0ed874c66cda1037a613cc845c4e7e/cf2 in testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e. 2023-05-05 15:53:56,458 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 6f0ed874c66cda1037a613cc845c4e7e/cf1 in testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e. 2023-05-05 15:53:56,459 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/635b6eae56774f39a70460b7ccb8d1aa.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/635b6eae56774f39a70460b7ccb8d1aa-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/c743bc8803c947088e8f55d598ec90ee.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/c743bc8803c947088e8f55d598ec90ee-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/8b37c8ad67154ac78eda03e1bd7a7efe.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8b37c8ad67154ac78eda03e1bd7a7efe-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/8d2decae984d45f3b4dd55c89443967f.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8d2decae984d45f3b4dd55c89443967f-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/d24ededcffd9484698b594c561b7ce37.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/d24ededcffd9484698b594c561b7ce37-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/952352901c844cc4bcf7b2a33b5c189e.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/952352901c844cc4bcf7b2a33b5c189e-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/1d51d0031db74e5fb0ec38addeb97226.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/1d51d0031db74e5fb0ec38addeb97226-bottom] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/.tmp, totalSize=34.9 K 2023-05-05 15:53:56,459 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/2dc4637c40d6410ab1613fd8199dc5d4.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/2dc4637c40d6410ab1613fd8199dc5d4-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/56ff2ff5709e493cb80be0ea26615515.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/56ff2ff5709e493cb80be0ea26615515-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/384cb59920554d27854b7194e9d15fcf.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/384cb59920554d27854b7194e9d15fcf-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/eb967395d76242878a77a4ad94aed787.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/eb967395d76242878a77a4ad94aed787-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/6f2500f0662b440683ac5ec2e44b19a9.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/6f2500f0662b440683ac5ec2e44b19a9-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/7bc278e48d844abda430f63025011cc7.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/7bc278e48d844abda430f63025011cc7-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/1a5bf8c567904f709a27885b51fb83bb.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/1a5bf8c567904f709a27885b51fb83bb-bottom] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/.tmp, totalSize=34.9 K 2023-05-05 15:53:56,459 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 635b6eae56774f39a70460b7ccb8d1aa.82d51cc6743c70579253652610cbc613, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683302031081 2023-05-05 15:53:56,460 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 2dc4637c40d6410ab1613fd8199dc5d4.82d51cc6743c70579253652610cbc613, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683302031081 2023-05-05 15:53:56,460 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e. 2023-05-05 15:53:56,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e. 2023-05-05 15:53:56,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564. 2023-05-05 15:53:56,460 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e800c4983abdf0cef46c4396eec67564, NAME => 'testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.', STARTKEY => '41', ENDKEY => ''} 2023-05-05 15:53:56,460 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting c743bc8803c947088e8f55d598ec90ee.82d51cc6743c70579253652610cbc613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683302031295 2023-05-05 15:53:56,460 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 56ff2ff5709e493cb80be0ea26615515.82d51cc6743c70579253652610cbc613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683302031295 2023-05-05 15:53:56,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:53:56,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:56,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:53:56,461 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 8b37c8ad67154ac78eda03e1bd7a7efe.82d51cc6743c70579253652610cbc613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683302031519 2023-05-05 15:53:56,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:56,461 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 384cb59920554d27854b7194e9d15fcf.82d51cc6743c70579253652610cbc613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683302031519 2023-05-05 15:53:56,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:56,461 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 8d2decae984d45f3b4dd55c89443967f.82d51cc6743c70579253652610cbc613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683302031735 2023-05-05 15:53:56,461 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=6f0ed874c66cda1037a613cc845c4e7e, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:56,461 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting eb967395d76242878a77a4ad94aed787.82d51cc6743c70579253652610cbc613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683302031735 2023-05-05 15:53:56,462 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683302036461"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302036461"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302036461"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302036461"}]},"ts":"1683302036461"} 2023-05-05 15:53:56,462 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting d24ededcffd9484698b594c561b7ce37.82d51cc6743c70579253652610cbc613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683302031943 2023-05-05 15:53:56,462 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 6f2500f0662b440683ac5ec2e44b19a9.82d51cc6743c70579253652610cbc613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683302031943 2023-05-05 15:53:56,462 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 952352901c844cc4bcf7b2a33b5c189e.82d51cc6743c70579253652610cbc613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683302032165 2023-05-05 15:53:56,462 INFO [StoreOpener-e800c4983abdf0cef46c4396eec67564-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:56,463 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 7bc278e48d844abda430f63025011cc7.82d51cc6743c70579253652610cbc613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683302032165 2023-05-05 15:53:56,463 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 1d51d0031db74e5fb0ec38addeb97226.82d51cc6743c70579253652610cbc613, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683302032366 2023-05-05 15:53:56,463 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 1a5bf8c567904f709a27885b51fb83bb.82d51cc6743c70579253652610cbc613, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683302032366 2023-05-05 15:53:56,464 DEBUG [StoreOpener-e800c4983abdf0cef46c4396eec67564-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1 2023-05-05 15:53:56,464 DEBUG [StoreOpener-e800c4983abdf0cef46c4396eec67564-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1 2023-05-05 15:53:56,465 INFO [StoreOpener-e800c4983abdf0cef46c4396eec67564-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e800c4983abdf0cef46c4396eec67564 columnFamilyName cf1 2023-05-05 15:53:56,466 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-05-05 15:53:56,466 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure 6f0ed874c66cda1037a613cc845c4e7e, server=jenkins-hbase4.apache.org,40991,1683302023016 in 266 msec 2023-05-05 15:53:56,468 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f0ed874c66cda1037a613cc845c4e7e, ASSIGN in 426 msec 2023-05-05 15:53:56,496 DEBUG [StoreOpener-e800c4983abdf0cef46c4396eec67564-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/1a5bf8c567904f709a27885b51fb83bb.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/1a5bf8c567904f709a27885b51fb83bb-top 2023-05-05 15:53:56,499 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] throttle.PressureAwareThroughputController(145): 6f0ed874c66cda1037a613cc845c4e7e#cf2#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:53:56,500 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6f0ed874c66cda1037a613cc845c4e7e#cf1#compaction#33 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:53:56,501 DEBUG [StoreOpener-e800c4983abdf0cef46c4396eec67564-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/2dc4637c40d6410ab1613fd8199dc5d4.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/2dc4637c40d6410ab1613fd8199dc5d4-top 2023-05-05 15:53:56,512 DEBUG [StoreOpener-e800c4983abdf0cef46c4396eec67564-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/384cb59920554d27854b7194e9d15fcf.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/384cb59920554d27854b7194e9d15fcf-top 2023-05-05 15:53:56,520 DEBUG [StoreOpener-e800c4983abdf0cef46c4396eec67564-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/56ff2ff5709e493cb80be0ea26615515.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/56ff2ff5709e493cb80be0ea26615515-top 2023-05-05 15:53:56,529 DEBUG [StoreOpener-e800c4983abdf0cef46c4396eec67564-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/6f2500f0662b440683ac5ec2e44b19a9.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/6f2500f0662b440683ac5ec2e44b19a9-top 2023-05-05 15:53:56,537 DEBUG [StoreOpener-e800c4983abdf0cef46c4396eec67564-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/7bc278e48d844abda430f63025011cc7.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/7bc278e48d844abda430f63025011cc7-top 2023-05-05 15:53:56,538 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/.tmp/cf1/a147d549e04b46a7b776095d98785291 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/a147d549e04b46a7b776095d98785291 2023-05-05 15:53:56,540 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/.tmp/cf2/9d677d8f827147bab9bb699ab001babc as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/9d677d8f827147bab9bb699ab001babc 2023-05-05 15:53:56,542 DEBUG [StoreOpener-e800c4983abdf0cef46c4396eec67564-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/eb967395d76242878a77a4ad94aed787.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/eb967395d76242878a77a4ad94aed787-top 2023-05-05 15:53:56,543 INFO [StoreOpener-e800c4983abdf0cef46c4396eec67564-1] regionserver.HStore(310): Store=e800c4983abdf0cef46c4396eec67564/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:53:56,543 INFO [StoreOpener-e800c4983abdf0cef46c4396eec67564-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:56,544 DEBUG [StoreOpener-e800c4983abdf0cef46c4396eec67564-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2 2023-05-05 15:53:56,544 DEBUG [StoreOpener-e800c4983abdf0cef46c4396eec67564-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2 2023-05-05 15:53:56,545 INFO [StoreOpener-e800c4983abdf0cef46c4396eec67564-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e800c4983abdf0cef46c4396eec67564 columnFamilyName cf2 2023-05-05 15:53:56,547 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6f0ed874c66cda1037a613cc845c4e7e/cf1 of 6f0ed874c66cda1037a613cc845c4e7e into a147d549e04b46a7b776095d98785291(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:53:56,547 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6f0ed874c66cda1037a613cc845c4e7e: 2023-05-05 15:53:56,547 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e., storeName=6f0ed874c66cda1037a613cc845c4e7e/cf1, priority=9, startTime=1683302036457; duration=0sec 2023-05-05 15:53:56,548 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:53:56,556 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6f0ed874c66cda1037a613cc845c4e7e/cf2 of 6f0ed874c66cda1037a613cc845c4e7e into 9d677d8f827147bab9bb699ab001babc(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:53:56,557 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6f0ed874c66cda1037a613cc845c4e7e: 2023-05-05 15:53:56,557 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e., storeName=6f0ed874c66cda1037a613cc845c4e7e/cf2, priority=9, startTime=1683302036457; duration=0sec 2023-05-05 15:53:56,557 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:53:56,566 DEBUG [StoreOpener-e800c4983abdf0cef46c4396eec67564-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/1d51d0031db74e5fb0ec38addeb97226.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/1d51d0031db74e5fb0ec38addeb97226-top 2023-05-05 15:53:56,570 DEBUG [StoreOpener-e800c4983abdf0cef46c4396eec67564-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/635b6eae56774f39a70460b7ccb8d1aa.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/635b6eae56774f39a70460b7ccb8d1aa-top 2023-05-05 15:53:56,574 DEBUG [StoreOpener-e800c4983abdf0cef46c4396eec67564-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/8b37c8ad67154ac78eda03e1bd7a7efe.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8b37c8ad67154ac78eda03e1bd7a7efe-top 2023-05-05 15:53:56,578 DEBUG [StoreOpener-e800c4983abdf0cef46c4396eec67564-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/8d2decae984d45f3b4dd55c89443967f.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8d2decae984d45f3b4dd55c89443967f-top 2023-05-05 15:53:56,582 DEBUG [StoreOpener-e800c4983abdf0cef46c4396eec67564-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/952352901c844cc4bcf7b2a33b5c189e.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/952352901c844cc4bcf7b2a33b5c189e-top 2023-05-05 15:53:56,589 DEBUG [StoreOpener-e800c4983abdf0cef46c4396eec67564-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/c743bc8803c947088e8f55d598ec90ee.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/c743bc8803c947088e8f55d598ec90ee-top 2023-05-05 15:53:56,593 DEBUG [StoreOpener-e800c4983abdf0cef46c4396eec67564-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/d24ededcffd9484698b594c561b7ce37.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/d24ededcffd9484698b594c561b7ce37-top 2023-05-05 15:53:56,593 INFO [StoreOpener-e800c4983abdf0cef46c4396eec67564-1] regionserver.HStore(310): Store=e800c4983abdf0cef46c4396eec67564/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:53:56,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:56,595 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:56,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-05 15:53:56,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:56,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e800c4983abdf0cef46c4396eec67564; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63496516, jitterRate=-0.05382817983627319}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-05 15:53:56,599 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e800c4983abdf0cef46c4396eec67564: 2023-05-05 15:53:56,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564., pid=34, masterSystemTime=1683302036350 2023-05-05 15:53:56,600 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:53:56,601 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-05 15:53:56,601 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-05 15:53:56,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-05 15:53:56,601 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564. 2023-05-05 15:53:56,601 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564. 2023-05-05 15:53:56,601 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1912): e800c4983abdf0cef46c4396eec67564/cf2 is initiating minor compaction (all files) 2023-05-05 15:53:56,601 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1912): e800c4983abdf0cef46c4396eec67564/cf1 is initiating minor compaction (all files) 2023-05-05 15:53:56,601 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegion(2259): Starting compaction of e800c4983abdf0cef46c4396eec67564/cf2 in testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564. 2023-05-05 15:53:56,601 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of e800c4983abdf0cef46c4396eec67564/cf1 in testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564. 2023-05-05 15:53:56,602 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/2dc4637c40d6410ab1613fd8199dc5d4.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/2dc4637c40d6410ab1613fd8199dc5d4-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/56ff2ff5709e493cb80be0ea26615515.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/56ff2ff5709e493cb80be0ea26615515-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/384cb59920554d27854b7194e9d15fcf.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/384cb59920554d27854b7194e9d15fcf-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/eb967395d76242878a77a4ad94aed787.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/eb967395d76242878a77a4ad94aed787-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/6f2500f0662b440683ac5ec2e44b19a9.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/6f2500f0662b440683ac5ec2e44b19a9-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/7bc278e48d844abda430f63025011cc7.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/7bc278e48d844abda430f63025011cc7-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/1a5bf8c567904f709a27885b51fb83bb.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/1a5bf8c567904f709a27885b51fb83bb-top] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/.tmp, totalSize=34.9 K 2023-05-05 15:53:56,602 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/635b6eae56774f39a70460b7ccb8d1aa.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/635b6eae56774f39a70460b7ccb8d1aa-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/c743bc8803c947088e8f55d598ec90ee.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/c743bc8803c947088e8f55d598ec90ee-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/8b37c8ad67154ac78eda03e1bd7a7efe.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8b37c8ad67154ac78eda03e1bd7a7efe-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/8d2decae984d45f3b4dd55c89443967f.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8d2decae984d45f3b4dd55c89443967f-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/d24ededcffd9484698b594c561b7ce37.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/d24ededcffd9484698b594c561b7ce37-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/952352901c844cc4bcf7b2a33b5c189e.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/952352901c844cc4bcf7b2a33b5c189e-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/1d51d0031db74e5fb0ec38addeb97226.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/1d51d0031db74e5fb0ec38addeb97226-top] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/.tmp, totalSize=34.9 K 2023-05-05 15:53:56,602 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 2dc4637c40d6410ab1613fd8199dc5d4.82d51cc6743c70579253652610cbc613, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683302031081 2023-05-05 15:53:56,603 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 635b6eae56774f39a70460b7ccb8d1aa.82d51cc6743c70579253652610cbc613, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683302031081 2023-05-05 15:53:56,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564. 2023-05-05 15:53:56,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564. 2023-05-05 15:53:56,603 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 56ff2ff5709e493cb80be0ea26615515.82d51cc6743c70579253652610cbc613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683302031295 2023-05-05 15:53:56,603 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting c743bc8803c947088e8f55d598ec90ee.82d51cc6743c70579253652610cbc613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683302031295 2023-05-05 15:53:56,603 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=e800c4983abdf0cef46c4396eec67564, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:56,604 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683302036603"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302036603"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302036603"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302036603"}]},"ts":"1683302036603"} 2023-05-05 15:53:56,604 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 384cb59920554d27854b7194e9d15fcf.82d51cc6743c70579253652610cbc613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683302031519 2023-05-05 15:53:56,604 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 8b37c8ad67154ac78eda03e1bd7a7efe.82d51cc6743c70579253652610cbc613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683302031519 2023-05-05 15:53:56,604 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting eb967395d76242878a77a4ad94aed787.82d51cc6743c70579253652610cbc613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1683302031735 2023-05-05 15:53:56,604 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 8d2decae984d45f3b4dd55c89443967f.82d51cc6743c70579253652610cbc613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1683302031735 2023-05-05 15:53:56,605 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 6f2500f0662b440683ac5ec2e44b19a9.82d51cc6743c70579253652610cbc613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1683302031943 2023-05-05 15:53:56,605 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting d24ededcffd9484698b594c561b7ce37.82d51cc6743c70579253652610cbc613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1683302031943 2023-05-05 15:53:56,606 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 7bc278e48d844abda430f63025011cc7.82d51cc6743c70579253652610cbc613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1683302032165 2023-05-05 15:53:56,606 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 952352901c844cc4bcf7b2a33b5c189e.82d51cc6743c70579253652610cbc613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1683302032165 2023-05-05 15:53:56,606 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 1a5bf8c567904f709a27885b51fb83bb.82d51cc6743c70579253652610cbc613, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683302032366 2023-05-05 15:53:56,606 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 1d51d0031db74e5fb0ec38addeb97226.82d51cc6743c70579253652610cbc613, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683302032366 2023-05-05 15:53:56,608 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-05-05 15:53:56,608 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure e800c4983abdf0cef46c4396eec67564, server=jenkins-hbase4.apache.org,40991,1683302023016 in 401 msec 2023-05-05 15:53:56,610 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=28 2023-05-05 15:53:56,611 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=e800c4983abdf0cef46c4396eec67564, ASSIGN in 567 msec 2023-05-05 15:53:56,613 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 in 3.9820 sec 2023-05-05 15:53:56,623 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] throttle.PressureAwareThroughputController(145): e800c4983abdf0cef46c4396eec67564#cf1#compaction#34 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:53:56,632 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] throttle.PressureAwareThroughputController(145): e800c4983abdf0cef46c4396eec67564#cf2#compaction#35 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:53:56,651 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/.tmp/cf1/4b2aa81d462e4a5abfa0b3b647c7efac as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/4b2aa81d462e4a5abfa0b3b647c7efac 2023-05-05 15:53:56,651 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/.tmp/cf2/8a34b1c8c9174c69b2330c92119539a1 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/8a34b1c8c9174c69b2330c92119539a1 2023-05-05 15:53:56,657 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e800c4983abdf0cef46c4396eec67564/cf1 of e800c4983abdf0cef46c4396eec67564 into 4b2aa81d462e4a5abfa0b3b647c7efac(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:53:56,657 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for e800c4983abdf0cef46c4396eec67564: 2023-05-05 15:53:56,657 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564., storeName=e800c4983abdf0cef46c4396eec67564/cf1, priority=9, startTime=1683302036600; duration=0sec 2023-05-05 15:53:56,657 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:53:56,658 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e800c4983abdf0cef46c4396eec67564/cf2 of e800c4983abdf0cef46c4396eec67564 into 8a34b1c8c9174c69b2330c92119539a1(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:53:56,658 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for e800c4983abdf0cef46c4396eec67564: 2023-05-05 15:53:56,658 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564., storeName=e800c4983abdf0cef46c4396eec67564/cf2, priority=9, startTime=1683302036601; duration=0sec 2023-05-05 15:53:56,658 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:53:56,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40991] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:60952 deadline: 1683302096785, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613. is not online on jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:56,899 DEBUG [Listener at localhost/35117] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-05 15:53:56,908 INFO [Listener at localhost/35117] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-05-05 15:53:56,910 DEBUG [Listener at localhost/35117] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-05 15:53:56,926 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-05 15:53:56,928 INFO [Listener at localhost/35117] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-05-05 15:53:56,928 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitWithoutPONR 2023-05-05 15:53:56,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:53:56,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-05-05 15:53:56,932 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302036932"}]},"ts":"1683302036932"} 2023-05-05 15:53:56,934 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-05-05 15:53:56,987 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-05-05 15:53:56,989 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f0ed874c66cda1037a613cc845c4e7e, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=e800c4983abdf0cef46c4396eec67564, UNASSIGN}] 2023-05-05 15:53:56,989 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=e800c4983abdf0cef46c4396eec67564, UNASSIGN 2023-05-05 15:53:56,990 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=e800c4983abdf0cef46c4396eec67564, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:56,990 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683302036990"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302036990"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302036990"}]},"ts":"1683302036990"} 2023-05-05 15:53:56,992 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure e800c4983abdf0cef46c4396eec67564, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:53:56,994 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f0ed874c66cda1037a613cc845c4e7e, UNASSIGN 2023-05-05 15:53:56,995 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=6f0ed874c66cda1037a613cc845c4e7e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:56,995 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683302036995"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302036995"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302036995"}]},"ts":"1683302036995"} 2023-05-05 15:53:56,997 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure 6f0ed874c66cda1037a613cc845c4e7e, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:53:57,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-05-05 15:53:57,144 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:57,144 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e800c4983abdf0cef46c4396eec67564, disabling compactions & flushes 2023-05-05 15:53:57,144 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564. 2023-05-05 15:53:57,144 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564. 2023-05-05 15:53:57,144 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564. after waiting 0 ms 2023-05-05 15:53:57,144 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564. 2023-05-05 15:53:57,161 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/2dc4637c40d6410ab1613fd8199dc5d4.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/2dc4637c40d6410ab1613fd8199dc5d4-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/56ff2ff5709e493cb80be0ea26615515.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/56ff2ff5709e493cb80be0ea26615515-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/384cb59920554d27854b7194e9d15fcf.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/384cb59920554d27854b7194e9d15fcf-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/eb967395d76242878a77a4ad94aed787.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/eb967395d76242878a77a4ad94aed787-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/6f2500f0662b440683ac5ec2e44b19a9.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/6f2500f0662b440683ac5ec2e44b19a9-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/7bc278e48d844abda430f63025011cc7.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/7bc278e48d844abda430f63025011cc7-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/1a5bf8c567904f709a27885b51fb83bb.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/1a5bf8c567904f709a27885b51fb83bb-top] to archive 2023-05-05 15:53:57,162 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-05 15:53:57,164 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/2dc4637c40d6410ab1613fd8199dc5d4.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/2dc4637c40d6410ab1613fd8199dc5d4.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,166 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/56ff2ff5709e493cb80be0ea26615515.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/56ff2ff5709e493cb80be0ea26615515.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,167 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/384cb59920554d27854b7194e9d15fcf.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/384cb59920554d27854b7194e9d15fcf.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,168 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/eb967395d76242878a77a4ad94aed787.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/eb967395d76242878a77a4ad94aed787.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,170 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/6f2500f0662b440683ac5ec2e44b19a9.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/6f2500f0662b440683ac5ec2e44b19a9.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,171 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/7bc278e48d844abda430f63025011cc7.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/7bc278e48d844abda430f63025011cc7.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,172 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/1a5bf8c567904f709a27885b51fb83bb.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/1a5bf8c567904f709a27885b51fb83bb.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,174 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/635b6eae56774f39a70460b7ccb8d1aa.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/635b6eae56774f39a70460b7ccb8d1aa-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/c743bc8803c947088e8f55d598ec90ee.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/c743bc8803c947088e8f55d598ec90ee-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/8b37c8ad67154ac78eda03e1bd7a7efe.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8b37c8ad67154ac78eda03e1bd7a7efe-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/8d2decae984d45f3b4dd55c89443967f.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8d2decae984d45f3b4dd55c89443967f-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/d24ededcffd9484698b594c561b7ce37.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/d24ededcffd9484698b594c561b7ce37-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/952352901c844cc4bcf7b2a33b5c189e.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/952352901c844cc4bcf7b2a33b5c189e-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/1d51d0031db74e5fb0ec38addeb97226.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/1d51d0031db74e5fb0ec38addeb97226-top] to archive 2023-05-05 15:53:57,175 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-05 15:53:57,177 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/635b6eae56774f39a70460b7ccb8d1aa.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/635b6eae56774f39a70460b7ccb8d1aa.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,178 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/c743bc8803c947088e8f55d598ec90ee.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/c743bc8803c947088e8f55d598ec90ee.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,179 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/8b37c8ad67154ac78eda03e1bd7a7efe.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/8b37c8ad67154ac78eda03e1bd7a7efe.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,181 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/8d2decae984d45f3b4dd55c89443967f.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/8d2decae984d45f3b4dd55c89443967f.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,182 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/d24ededcffd9484698b594c561b7ce37.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/d24ededcffd9484698b594c561b7ce37.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,183 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/952352901c844cc4bcf7b2a33b5c189e.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/952352901c844cc4bcf7b2a33b5c189e.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,184 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/1d51d0031db74e5fb0ec38addeb97226.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/1d51d0031db74e5fb0ec38addeb97226.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,188 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-05 15:53:57,188 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:53:57,189 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564. 2023-05-05 15:53:57,189 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e800c4983abdf0cef46c4396eec67564: 2023-05-05 15:53:57,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:57,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 6f0ed874c66cda1037a613cc845c4e7e 2023-05-05 15:53:57,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6f0ed874c66cda1037a613cc845c4e7e, disabling compactions & flushes 2023-05-05 15:53:57,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e. 2023-05-05 15:53:57,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e. 2023-05-05 15:53:57,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e. after waiting 0 ms 2023-05-05 15:53:57,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e. 2023-05-05 15:53:57,191 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=e800c4983abdf0cef46c4396eec67564, regionState=CLOSED 2023-05-05 15:53:57,191 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683302037191"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302037191"}]},"ts":"1683302037191"} 2023-05-05 15:53:57,191 DEBUG [StoreCloser-testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/2dc4637c40d6410ab1613fd8199dc5d4.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/2dc4637c40d6410ab1613fd8199dc5d4-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/56ff2ff5709e493cb80be0ea26615515.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/56ff2ff5709e493cb80be0ea26615515-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/384cb59920554d27854b7194e9d15fcf.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/384cb59920554d27854b7194e9d15fcf-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/eb967395d76242878a77a4ad94aed787.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/eb967395d76242878a77a4ad94aed787-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/6f2500f0662b440683ac5ec2e44b19a9.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/6f2500f0662b440683ac5ec2e44b19a9-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/7bc278e48d844abda430f63025011cc7.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/7bc278e48d844abda430f63025011cc7-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/1a5bf8c567904f709a27885b51fb83bb.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/1a5bf8c567904f709a27885b51fb83bb-bottom] to archive 2023-05-05 15:53:57,192 DEBUG [StoreCloser-testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-05 15:53:57,194 DEBUG [StoreCloser-testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/2dc4637c40d6410ab1613fd8199dc5d4.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/2dc4637c40d6410ab1613fd8199dc5d4.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,196 DEBUG [StoreCloser-testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/56ff2ff5709e493cb80be0ea26615515.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/56ff2ff5709e493cb80be0ea26615515.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,197 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-05-05 15:53:57,197 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure e800c4983abdf0cef46c4396eec67564, server=jenkins-hbase4.apache.org,40991,1683302023016 in 202 msec 2023-05-05 15:53:57,198 DEBUG [StoreCloser-testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/384cb59920554d27854b7194e9d15fcf.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/384cb59920554d27854b7194e9d15fcf.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,199 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=e800c4983abdf0cef46c4396eec67564, UNASSIGN in 208 msec 2023-05-05 15:53:57,204 DEBUG [StoreCloser-testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/eb967395d76242878a77a4ad94aed787.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/eb967395d76242878a77a4ad94aed787.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,206 DEBUG [StoreCloser-testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/6f2500f0662b440683ac5ec2e44b19a9.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/6f2500f0662b440683ac5ec2e44b19a9.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,207 DEBUG [StoreCloser-testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/7bc278e48d844abda430f63025011cc7.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/7bc278e48d844abda430f63025011cc7.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,208 DEBUG [StoreCloser-testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/1a5bf8c567904f709a27885b51fb83bb.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/1a5bf8c567904f709a27885b51fb83bb.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,210 DEBUG [StoreCloser-testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/635b6eae56774f39a70460b7ccb8d1aa.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/635b6eae56774f39a70460b7ccb8d1aa-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/c743bc8803c947088e8f55d598ec90ee.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/c743bc8803c947088e8f55d598ec90ee-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/8b37c8ad67154ac78eda03e1bd7a7efe.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8b37c8ad67154ac78eda03e1bd7a7efe-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/8d2decae984d45f3b4dd55c89443967f.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8d2decae984d45f3b4dd55c89443967f-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/d24ededcffd9484698b594c561b7ce37.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/d24ededcffd9484698b594c561b7ce37-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/952352901c844cc4bcf7b2a33b5c189e.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/952352901c844cc4bcf7b2a33b5c189e-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/1d51d0031db74e5fb0ec38addeb97226.82d51cc6743c70579253652610cbc613->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/1d51d0031db74e5fb0ec38addeb97226-bottom] to archive 2023-05-05 15:53:57,211 DEBUG [StoreCloser-testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-05 15:53:57,212 DEBUG [StoreCloser-testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/635b6eae56774f39a70460b7ccb8d1aa.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/635b6eae56774f39a70460b7ccb8d1aa.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,214 DEBUG [StoreCloser-testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/c743bc8803c947088e8f55d598ec90ee.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/c743bc8803c947088e8f55d598ec90ee.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,215 DEBUG [StoreCloser-testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/8b37c8ad67154ac78eda03e1bd7a7efe.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/8b37c8ad67154ac78eda03e1bd7a7efe.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,216 DEBUG [StoreCloser-testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/8d2decae984d45f3b4dd55c89443967f.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/8d2decae984d45f3b4dd55c89443967f.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,217 DEBUG [StoreCloser-testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/d24ededcffd9484698b594c561b7ce37.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/d24ededcffd9484698b594c561b7ce37.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,218 DEBUG [StoreCloser-testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/952352901c844cc4bcf7b2a33b5c189e.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/952352901c844cc4bcf7b2a33b5c189e.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,220 DEBUG [StoreCloser-testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/1d51d0031db74e5fb0ec38addeb97226.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/1d51d0031db74e5fb0ec38addeb97226.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,223 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-05 15:53:57,224 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:53:57,224 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e. 2023-05-05 15:53:57,224 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6f0ed874c66cda1037a613cc845c4e7e: 2023-05-05 15:53:57,226 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 6f0ed874c66cda1037a613cc845c4e7e 2023-05-05 15:53:57,226 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=6f0ed874c66cda1037a613cc845c4e7e, regionState=CLOSED 2023-05-05 15:53:57,226 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683302037226"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302037226"}]},"ts":"1683302037226"} 2023-05-05 15:53:57,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-05-05 15:53:57,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure 6f0ed874c66cda1037a613cc845c4e7e, server=jenkins-hbase4.apache.org,40991,1683302023016 in 231 msec 2023-05-05 15:53:57,233 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-05-05 15:53:57,233 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f0ed874c66cda1037a613cc845c4e7e, UNASSIGN in 242 msec 2023-05-05 15:53:57,234 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302037234"}]},"ts":"1683302037234"} 2023-05-05 15:53:57,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-05-05 15:53:57,235 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-05-05 15:53:57,310 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-05-05 15:53:57,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 383 msec 2023-05-05 15:53:57,536 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-05-05 15:53:57,536 INFO [Listener at localhost/35117] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-05-05 15:53:57,537 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitWithoutPONR 2023-05-05 15:53:57,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:53:57,540 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:53:57,540 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:53:57,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-05-05 15:53:57,548 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,548 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:57,548 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e 2023-05-05 15:53:57,551 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/recovered.edits] 2023-05-05 15:53:57,551 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/recovered.edits] 2023-05-05 15:53:57,551 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/recovered.edits] 2023-05-05 15:53:57,554 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/4b2aa81d462e4a5abfa0b3b647c7efac to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf1/4b2aa81d462e4a5abfa0b3b647c7efac 2023-05-05 15:53:57,555 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/1a5bf8c567904f709a27885b51fb83bb to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/1a5bf8c567904f709a27885b51fb83bb 2023-05-05 15:53:57,555 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/.regioninfo to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/.regioninfo 2023-05-05 15:53:57,557 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/2dc4637c40d6410ab1613fd8199dc5d4 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/2dc4637c40d6410ab1613fd8199dc5d4 2023-05-05 15:53:57,557 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/8a34b1c8c9174c69b2330c92119539a1 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/cf2/8a34b1c8c9174c69b2330c92119539a1 2023-05-05 15:53:57,559 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/384cb59920554d27854b7194e9d15fcf to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/384cb59920554d27854b7194e9d15fcf 2023-05-05 15:53:57,560 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf1/1a5bf8c567904f709a27885b51fb83bb.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf1/1a5bf8c567904f709a27885b51fb83bb.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,561 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/56ff2ff5709e493cb80be0ea26615515 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/56ff2ff5709e493cb80be0ea26615515 2023-05-05 15:53:57,561 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/.regioninfo to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/.regioninfo 2023-05-05 15:53:57,562 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf1/2dc4637c40d6410ab1613fd8199dc5d4.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf1/2dc4637c40d6410ab1613fd8199dc5d4.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,563 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/6f2500f0662b440683ac5ec2e44b19a9 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/6f2500f0662b440683ac5ec2e44b19a9 2023-05-05 15:53:57,564 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf1/384cb59920554d27854b7194e9d15fcf.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf1/384cb59920554d27854b7194e9d15fcf.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,565 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/7bc278e48d844abda430f63025011cc7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/7bc278e48d844abda430f63025011cc7 2023-05-05 15:53:57,565 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf1/1a5bf8c567904f709a27885b51fb83bb.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf1/1a5bf8c567904f709a27885b51fb83bb.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,566 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf1/56ff2ff5709e493cb80be0ea26615515.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf1/56ff2ff5709e493cb80be0ea26615515.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,567 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/eb967395d76242878a77a4ad94aed787 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf1/eb967395d76242878a77a4ad94aed787 2023-05-05 15:53:57,567 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf1/2dc4637c40d6410ab1613fd8199dc5d4.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf1/2dc4637c40d6410ab1613fd8199dc5d4.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,568 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf1/6f2500f0662b440683ac5ec2e44b19a9.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf1/6f2500f0662b440683ac5ec2e44b19a9.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,569 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf1/384cb59920554d27854b7194e9d15fcf.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf1/384cb59920554d27854b7194e9d15fcf.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,570 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf1/7bc278e48d844abda430f63025011cc7.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf1/7bc278e48d844abda430f63025011cc7.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,571 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf1/56ff2ff5709e493cb80be0ea26615515.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf1/56ff2ff5709e493cb80be0ea26615515.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,571 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/1d51d0031db74e5fb0ec38addeb97226 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/1d51d0031db74e5fb0ec38addeb97226 2023-05-05 15:53:57,572 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf1/eb967395d76242878a77a4ad94aed787.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf1/eb967395d76242878a77a4ad94aed787.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,573 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf1/6f2500f0662b440683ac5ec2e44b19a9.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf1/6f2500f0662b440683ac5ec2e44b19a9.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,574 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/635b6eae56774f39a70460b7ccb8d1aa to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/635b6eae56774f39a70460b7ccb8d1aa 2023-05-05 15:53:57,582 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf1/7bc278e48d844abda430f63025011cc7.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf1/7bc278e48d844abda430f63025011cc7.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,583 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8b37c8ad67154ac78eda03e1bd7a7efe to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8b37c8ad67154ac78eda03e1bd7a7efe 2023-05-05 15:53:57,584 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf2/1d51d0031db74e5fb0ec38addeb97226.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf2/1d51d0031db74e5fb0ec38addeb97226.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,585 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf1/eb967395d76242878a77a4ad94aed787.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf1/eb967395d76242878a77a4ad94aed787.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,585 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8d2decae984d45f3b4dd55c89443967f to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/8d2decae984d45f3b4dd55c89443967f 2023-05-05 15:53:57,587 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/952352901c844cc4bcf7b2a33b5c189e to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/952352901c844cc4bcf7b2a33b5c189e 2023-05-05 15:53:57,587 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf2/635b6eae56774f39a70460b7ccb8d1aa.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf2/635b6eae56774f39a70460b7ccb8d1aa.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,592 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/c743bc8803c947088e8f55d598ec90ee to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/c743bc8803c947088e8f55d598ec90ee 2023-05-05 15:53:57,593 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf2/8b37c8ad67154ac78eda03e1bd7a7efe.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf2/8b37c8ad67154ac78eda03e1bd7a7efe.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,595 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/d24ededcffd9484698b594c561b7ce37 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/cf2/d24ededcffd9484698b594c561b7ce37 2023-05-05 15:53:57,597 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf2/8d2decae984d45f3b4dd55c89443967f.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf2/8d2decae984d45f3b4dd55c89443967f.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,597 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf2/1d51d0031db74e5fb0ec38addeb97226.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf2/1d51d0031db74e5fb0ec38addeb97226.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,600 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf2/952352901c844cc4bcf7b2a33b5c189e.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf2/952352901c844cc4bcf7b2a33b5c189e.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,601 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf2/635b6eae56774f39a70460b7ccb8d1aa.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf2/635b6eae56774f39a70460b7ccb8d1aa.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,605 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/recovered.edits/85.seqid to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613/recovered.edits/85.seqid 2023-05-05 15:53:57,606 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,606 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf2/c743bc8803c947088e8f55d598ec90ee.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf2/c743bc8803c947088e8f55d598ec90ee.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,607 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf2/8b37c8ad67154ac78eda03e1bd7a7efe.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf2/8b37c8ad67154ac78eda03e1bd7a7efe.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,608 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf2/d24ededcffd9484698b594c561b7ce37.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/6f0ed874c66cda1037a613cc845c4e7e/cf2/d24ededcffd9484698b594c561b7ce37.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,608 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf2/8d2decae984d45f3b4dd55c89443967f.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf2/8d2decae984d45f3b4dd55c89443967f.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,610 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf2/952352901c844cc4bcf7b2a33b5c189e.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf2/952352901c844cc4bcf7b2a33b5c189e.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,611 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/a147d549e04b46a7b776095d98785291 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf1/a147d549e04b46a7b776095d98785291 2023-05-05 15:53:57,612 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf2/c743bc8803c947088e8f55d598ec90ee.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf2/c743bc8803c947088e8f55d598ec90ee.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,613 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf2/d24ededcffd9484698b594c561b7ce37.82d51cc6743c70579253652610cbc613 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/e800c4983abdf0cef46c4396eec67564/cf2/d24ededcffd9484698b594c561b7ce37.82d51cc6743c70579253652610cbc613 2023-05-05 15:53:57,615 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/9d677d8f827147bab9bb699ab001babc to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/cf2/9d677d8f827147bab9bb699ab001babc 2023-05-05 15:53:57,617 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/recovered.edits/90.seqid to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564/recovered.edits/90.seqid 2023-05-05 15:53:57,618 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/e800c4983abdf0cef46c4396eec67564 2023-05-05 15:53:57,620 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/recovered.edits/90.seqid to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e/recovered.edits/90.seqid 2023-05-05 15:53:57,620 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitWithoutPONR/6f0ed874c66cda1037a613cc845c4e7e 2023-05-05 15:53:57,621 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-05-05 15:53:57,623 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:53:57,626 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-05-05 15:53:57,629 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-05-05 15:53:57,630 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:53:57,631 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-05-05 15:53:57,631 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302037631"}]},"ts":"9223372036854775807"} 2023-05-05 15:53:57,631 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302037631"}]},"ts":"9223372036854775807"} 2023-05-05 15:53:57,631 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302037631"}]},"ts":"9223372036854775807"} 2023-05-05 15:53:57,633 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-05 15:53:57,633 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 82d51cc6743c70579253652610cbc613, NAME => 'testSplitWithoutPONR,,1683302030575.82d51cc6743c70579253652610cbc613.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 6f0ed874c66cda1037a613cc845c4e7e, NAME => 'testSplitWithoutPONR,,1683302032629.6f0ed874c66cda1037a613cc845c4e7e.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => e800c4983abdf0cef46c4396eec67564, NAME => 'testSplitWithoutPONR,41,1683302032629.e800c4983abdf0cef46c4396eec67564.', STARTKEY => '41', ENDKEY => ''}] 2023-05-05 15:53:57,633 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-05-05 15:53:57,633 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683302037633"}]},"ts":"9223372036854775807"} 2023-05-05 15:53:57,634 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-05-05 15:53:57,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-05-05 15:53:57,666 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:53:57,667 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 129 msec 2023-05-05 15:53:57,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-05-05 15:53:57,843 INFO [Listener at localhost/35117] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-05-05 15:53:57,858 INFO [Listener at localhost/35117] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=402 (was 394) Potentially hanging thread: RS:2;jenkins-hbase4:40991-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:55052 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:46556 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:47242 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:55080 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:35098 [Waiting for operation #52] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:54668 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:55076 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/cluster_ed487d35-94af-afba-80b3-40ed4e353473/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/cluster_ed487d35-94af-afba-80b3-40ed4e353473/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:54532 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:36028 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6277894c-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x603c4d85-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/cluster_ed487d35-94af-afba-80b3-40ed4e353473/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:47108 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:54712 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:36068 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6277894c-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:54918 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x603c4d85-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6277894c-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/cluster_ed487d35-94af-afba-80b3-40ed4e353473/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6277894c-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6277894c-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:39988 [Waiting for operation #34] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6277894c-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:35016 [Waiting for operation #40] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:47200 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=868 (was 864) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=335 (was 347), ProcessCount=179 (was 179), AvailableMemoryMB=5316 (was 5317) 2023-05-05 15:53:57,873 INFO [Listener at localhost/35117] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=402, OpenFileDescriptor=868, MaxFileDescriptor=60000, SystemLoadAverage=335, ProcessCount=179, AvailableMemoryMB=5315 2023-05-05 15:53:57,873 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-05 15:53:57,876 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-05 15:53:57,876 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-05 15:53:57,878 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:53:57,879 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-05-05 15:53:57,880 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-05 15:53:57,882 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:53:57,882 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3 empty. 2023-05-05 15:53:57,883 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:53:57,883 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-05-05 15:53:57,899 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-05-05 15:53:57,900 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => b299eb879fd7ed0f63e4b91ba5268da3, NAME => 'testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp 2023-05-05 15:53:57,909 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:53:57,909 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing b299eb879fd7ed0f63e4b91ba5268da3, disabling compactions & flushes 2023-05-05 15:53:57,909 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:53:57,909 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:53:57,909 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. after waiting 0 ms 2023-05-05 15:53:57,909 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:53:57,909 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:53:57,909 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for b299eb879fd7ed0f63e4b91ba5268da3: 2023-05-05 15:53:57,912 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-05-05 15:53:57,913 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683302037913"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302037913"}]},"ts":"1683302037913"} 2023-05-05 15:53:57,914 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-05 15:53:57,915 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-05 15:53:57,916 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302037915"}]},"ts":"1683302037915"} 2023-05-05 15:53:57,917 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-05-05 15:53:57,920 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-05 15:53:57,921 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-05 15:53:57,921 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-05 15:53:57,921 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-05 15:53:57,921 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-05 15:53:57,921 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN}] 2023-05-05 15:53:57,923 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:53:57,924 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42799,1683302022977; forceNewPlan=false, retain=false 2023-05-05 15:53:58,074 INFO [jenkins-hbase4:44213] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-05 15:53:58,075 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=b299eb879fd7ed0f63e4b91ba5268da3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,076 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683302038075"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302038075"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302038075"}]},"ts":"1683302038075"} 2023-05-05 15:53:58,077 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977}] 2023-05-05 15:53:58,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:53:58,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b299eb879fd7ed0f63e4b91ba5268da3, NAME => 'testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.', STARTKEY => '', ENDKEY => ''} 2023-05-05 15:53:58,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:53:58,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:53:58,234 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:53:58,234 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:53:58,234 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:53:58,235 INFO [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:53:58,236 DEBUG [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1 2023-05-05 15:53:58,236 DEBUG [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1 2023-05-05 15:53:58,237 INFO [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b299eb879fd7ed0f63e4b91ba5268da3 columnFamilyName cf1 2023-05-05 15:53:58,237 INFO [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] regionserver.HStore(310): Store=b299eb879fd7ed0f63e4b91ba5268da3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:53:58,237 INFO [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:53:58,239 DEBUG [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2 2023-05-05 15:53:58,239 DEBUG [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2 2023-05-05 15:53:58,239 INFO [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b299eb879fd7ed0f63e4b91ba5268da3 columnFamilyName cf2 2023-05-05 15:53:58,240 INFO [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] regionserver.HStore(310): Store=b299eb879fd7ed0f63e4b91ba5268da3/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:53:58,240 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:53:58,241 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:53:58,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-05 15:53:58,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:53:58,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-05 15:53:58,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b299eb879fd7ed0f63e4b91ba5268da3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66580528, jitterRate=-0.007872819900512695}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-05 15:53:58,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b299eb879fd7ed0f63e4b91ba5268da3: 2023-05-05 15:53:58,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3., pid=43, masterSystemTime=1683302038229 2023-05-05 15:53:58,248 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:53:58,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:53:58,249 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=b299eb879fd7ed0f63e4b91ba5268da3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,249 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683302038249"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302038249"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302038249"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302038249"}]},"ts":"1683302038249"} 2023-05-05 15:53:58,252 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-05-05 15:53:58,252 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 in 173 msec 2023-05-05 15:53:58,254 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-05-05 15:53:58,254 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN in 331 msec 2023-05-05 15:53:58,255 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-05 15:53:58,255 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302038255"}]},"ts":"1683302038255"} 2023-05-05 15:53:58,256 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-05-05 15:53:58,263 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-05-05 15:53:58,265 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 386 msec 2023-05-05 15:53:58,387 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:53:58,389 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-05-05 15:53:58,389 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-05-05 15:53:58,389 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:58,389 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-05-05 15:53:58,389 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-05-05 15:53:58,389 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,390 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,391 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:58,391 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:58,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:58,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:58,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:58,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:58,391 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,391 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:58,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:58,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,391 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,392 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:53:58,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:58,392 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,392 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:58,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:58,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,392 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,392 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-05 15:53:58,392 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-05 15:53:58,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:58,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,393 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-05 15:53:58,393 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-05 15:53:58,393 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-05 15:53:58,393 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-05 15:53:58,393 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-05 15:53:58,393 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:58,393 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:58,393 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:53:58,393 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:53:58,393 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-05 15:53:58,393 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-05 15:53:58,393 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-05 15:53:58,393 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:53:58,393 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:53:58,393 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:53:58,393 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-05 15:53:58,393 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:58,393 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. started... 2023-05-05 15:53:58,394 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:53:58,394 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-05 15:53:58,394 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-05 15:53:58,395 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b299eb879fd7ed0f63e4b91ba5268da3 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-05 15:53:58,395 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,395 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,395 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:58,396 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,396 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,396 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:58,406 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf1/9eac5d62b79f49159581b82e191d6f4a 2023-05-05 15:53:58,422 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf2/1ea30cc7aa0f49b4bb29289547c266bb 2023-05-05 15:53:58,427 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf1/9eac5d62b79f49159581b82e191d6f4a as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/9eac5d62b79f49159581b82e191d6f4a 2023-05-05 15:53:58,432 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/9eac5d62b79f49159581b82e191d6f4a, entries=2, sequenceid=6, filesize=4.8 K 2023-05-05 15:53:58,433 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf2/1ea30cc7aa0f49b4bb29289547c266bb as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/1ea30cc7aa0f49b4bb29289547c266bb 2023-05-05 15:53:58,438 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/1ea30cc7aa0f49b4bb29289547c266bb, entries=2, sequenceid=6, filesize=4.8 K 2023-05-05 15:53:58,439 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for b299eb879fd7ed0f63e4b91ba5268da3 in 44ms, sequenceid=6, compaction requested=false 2023-05-05 15:53:58,439 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-05-05 15:53:58,440 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b299eb879fd7ed0f63e4b91ba5268da3: 2023-05-05 15:53:58,440 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:53:58,440 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:53:58,440 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:53:58,440 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,440 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-05 15:53:58,440 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-05 15:53:58,443 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,443 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:58,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:58,443 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,443 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:58,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:58,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:58,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:58,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:58,446 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-05-05 15:53:58,446 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-05-05 15:53:58,446 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a3a82e9[Count = 0] remaining members to acquire global barrier 2023-05-05 15:53:58,446 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,447 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,447 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,447 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,448 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:53:58,448 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:53:58,448 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-05 15:53:58,448 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,448 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:53:58,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,448 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:53:58,448 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-05 15:53:58,448 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:53:58,448 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:53:58,448 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-05 15:53:58,448 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:53:58,450 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-05 15:53:58,450 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:58,450 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-05 15:53:58,450 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-05 15:53:58,450 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,450 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:58,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:58,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:58,450 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-05 15:53:58,450 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-05 15:53:58,451 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:58,451 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-05 15:53:58,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:58,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:58,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:58,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:58,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:58,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,42799,1683302022977': 2023-05-05 15:53:58,455 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-05 15:53:58,455 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-05-05 15:53:58,455 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:53:58,455 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:53:58,455 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-05-05 15:53:58,455 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:53:58,458 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,458 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:58,458 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:58,458 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,458 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,458 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:58,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:58,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,459 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,458 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:58,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:58,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:58,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:58,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:58,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:58,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:58,460 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:58,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:58,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,460 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:58,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:58,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,462 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:58,462 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:58,463 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,466 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:58,466 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:58,466 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:58,466 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:58,466 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:58,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:58,466 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:58,466 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:58,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:58,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:58,466 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:58,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:58,467 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,466 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-05-05 15:53:58,467 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:53:58,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:58,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:58,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:58,467 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,467 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,467 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-05 15:53:58,467 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:58,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:58,467 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,467 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:53:58,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:58,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:58,467 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,468 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,468 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,468 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,468 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,568 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:53:58,569 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:53:58,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:53:58,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-05 15:53:58,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-05-05 15:53:58,588 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-05-05 15:53:58,588 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:58,588 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-05-05 15:53:58,588 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-05-05 15:53:58,588 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,588 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,590 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:58,590 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:58,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:58,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:58,590 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:58,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:58,590 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:58,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:58,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:58,590 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,591 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:53:58,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,591 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,591 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,591 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:58,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:58,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,592 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-05 15:53:58,592 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-05 15:53:58,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:58,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,592 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-05 15:53:58,592 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-05 15:53:58,592 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-05 15:53:58,592 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-05 15:53:58,592 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-05 15:53:58,592 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:58,592 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-05 15:53:58,592 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:53:58,593 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-05 15:53:58,593 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:58,593 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:53:58,593 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-05 15:53:58,593 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-05 15:53:58,593 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-05 15:53:58,593 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-05 15:53:58,593 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-05 15:53:58,593 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:58,593 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:53:58,593 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-05 15:53:58,593 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-05 15:53:58,595 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,597 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,597 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,597 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:58,597 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,597 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:58,600 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:53:58,600 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:53:58,600 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:53:58,600 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. started... 2023-05-05 15:53:58,600 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b299eb879fd7ed0f63e4b91ba5268da3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:53:58,613 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf1/a79dfb5adfe544bd973d9313c282fdcb 2023-05-05 15:53:58,631 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf2/8502d9b6ae874806a9213be374c9565d 2023-05-05 15:53:58,636 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf1/a79dfb5adfe544bd973d9313c282fdcb as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/a79dfb5adfe544bd973d9313c282fdcb 2023-05-05 15:53:58,641 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/a79dfb5adfe544bd973d9313c282fdcb, entries=10, sequenceid=19, filesize=5.0 K 2023-05-05 15:53:58,642 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf2/8502d9b6ae874806a9213be374c9565d as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/8502d9b6ae874806a9213be374c9565d 2023-05-05 15:53:58,648 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/8502d9b6ae874806a9213be374c9565d, entries=10, sequenceid=19, filesize=5.0 K 2023-05-05 15:53:58,650 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b299eb879fd7ed0f63e4b91ba5268da3 in 49ms, sequenceid=19, compaction requested=false 2023-05-05 15:53:58,650 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b299eb879fd7ed0f63e4b91ba5268da3: 2023-05-05 15:53:58,650 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:53:58,650 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:53:58,650 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:53:58,650 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,650 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-05 15:53:58,650 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-05 15:53:58,652 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,652 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:58,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:58,652 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,652 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:58,655 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-05-05 15:53:58,655 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7a3d6000[Count = 0] remaining members to acquire global barrier 2023-05-05 15:53:58,655 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-05-05 15:53:58,655 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,656 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,656 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,656 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,656 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:53:58,656 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:53:58,656 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-05 15:53:58,656 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,657 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:53:58,657 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:53:58,656 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-05 15:53:58,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,657 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:53:58,657 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:53:58,657 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-05 15:53:58,657 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:53:58,658 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-05 15:53:58,658 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,658 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:58,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:58,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:58,658 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-05 15:53:58,662 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-05 15:53:58,662 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-05 15:53:58,662 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:58,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:58,662 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-05 15:53:58,662 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:58,662 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-05 15:53:58,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:58,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:58,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:58,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:58,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,42799,1683302022977': 2023-05-05 15:53:58,666 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-05 15:53:58,666 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-05-05 15:53:58,666 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:53:58,666 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:53:58,666 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-05-05 15:53:58,666 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:53:58,668 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,668 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,668 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,668 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:58,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:58,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,668 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:58,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,668 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:58,668 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:58,668 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:58,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:58,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:58,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:58,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,669 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:58,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:58,669 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:58,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:58,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:58,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:58,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,671 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,672 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:58,672 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:58,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,675 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:58,675 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:58,675 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:58,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:58,675 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,675 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:58,675 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:58,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:58,675 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:58,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:58,675 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-05-05 15:53:58,676 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:53:58,675 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:58,675 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:58,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:58,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:58,676 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-05 15:53:58,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:58,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:58,676 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:53:58,676 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:58,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:58,676 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,677 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,677 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,677 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,677 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,677 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,677 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,677 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:58,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:58,776 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:53:58,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:53:58,793 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:53:58,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-05 15:53:58,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-05-05 15:53:58,794 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-05-05 15:53:58,794 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:58,794 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-05-05 15:53:58,794 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-05-05 15:53:58,795 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,795 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,796 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,796 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:58,796 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:58,796 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:58,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:58,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:58,796 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,796 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:53:58,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:58,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:58,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:58,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:58,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,797 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,797 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,797 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:58,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:58,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:58,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,798 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-05 15:53:58,798 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-05 15:53:58,798 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-05 15:53:58,798 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-05 15:53:58,798 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-05 15:53:58,798 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-05 15:53:58,798 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-05 15:53:58,798 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-05 15:53:58,798 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-05 15:53:58,798 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-05 15:53:58,798 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-05 15:53:58,798 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-05 15:53:58,798 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:58,798 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:58,798 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:58,798 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:53:58,798 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:53:58,799 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-05 15:53:58,799 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:53:58,799 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-05 15:53:58,799 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-05 15:53:58,799 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-05 15:53:58,799 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:53:58,799 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:53:58,799 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. started... 2023-05-05 15:53:58,799 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:53:58,799 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b299eb879fd7ed0f63e4b91ba5268da3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:53:58,801 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,802 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,802 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,803 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:58,803 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,803 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:58,816 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf1/9ce680e0fa604098a7aa8ac4b9b7166c 2023-05-05 15:53:58,830 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf2/fd1f211bab184c0a9df23ab9a2232ea2 2023-05-05 15:53:58,834 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf1/9ce680e0fa604098a7aa8ac4b9b7166c as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/9ce680e0fa604098a7aa8ac4b9b7166c 2023-05-05 15:53:58,839 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/9ce680e0fa604098a7aa8ac4b9b7166c, entries=10, sequenceid=32, filesize=5.0 K 2023-05-05 15:53:58,839 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf2/fd1f211bab184c0a9df23ab9a2232ea2 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/fd1f211bab184c0a9df23ab9a2232ea2 2023-05-05 15:53:58,844 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/fd1f211bab184c0a9df23ab9a2232ea2, entries=10, sequenceid=32, filesize=5.0 K 2023-05-05 15:53:58,844 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b299eb879fd7ed0f63e4b91ba5268da3 in 45ms, sequenceid=32, compaction requested=true 2023-05-05 15:53:58,845 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b299eb879fd7ed0f63e4b91ba5268da3: 2023-05-05 15:53:58,845 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:53:58,845 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:53:58,845 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:53:58,845 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,845 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-05 15:53:58,845 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-05 15:53:58,846 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,846 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:58,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:58,847 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,847 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:58,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:58,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:58,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:58,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:58,849 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-05-05 15:53:58,849 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b8af0ea[Count = 0] remaining members to acquire global barrier 2023-05-05 15:53:58,849 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-05-05 15:53:58,849 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,850 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,850 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,850 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,850 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,851 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:53:58,850 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:53:58,851 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-05 15:53:58,850 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:53:58,851 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:53:58,851 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-05 15:53:58,851 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:53:58,851 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-05 15:53:58,851 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:53:58,851 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:53:58,854 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-05 15:53:58,854 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-05 15:53:58,854 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-05 15:53:58,854 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:58,854 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:58,854 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-05 15:53:58,854 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:58,854 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-05 15:53:58,854 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-05 15:53:58,860 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:58,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:58,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:58,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:58,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:58,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:58,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:58,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,42799,1683302022977': 2023-05-05 15:53:58,863 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-05 15:53:58,863 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-05-05 15:53:58,863 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:53:58,863 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:53:58,863 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-05-05 15:53:58,863 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:53:58,865 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,865 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,865 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,865 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:58,865 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:58,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:58,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,865 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,865 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:58,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,865 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:58,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:58,866 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:58,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:58,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:58,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:58,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:58,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:58,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:58,866 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:58,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:58,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,868 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:58,869 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:58,869 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,875 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:58,875 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:58,875 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:58,875 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:58,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:58,875 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:58,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:58,876 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-05-05 15:53:58,876 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:53:58,875 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:58,876 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:58,875 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:58,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:58,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:58,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:58,876 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:58,876 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,876 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,876 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-05 15:53:58,876 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:53:58,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:58,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:58,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:58,876 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,877 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:58,877 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,877 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:58,877 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,877 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:58,877 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:58,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:58,977 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:53:58,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:53:58,993 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:53:58,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-05 15:53:58,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-05-05 15:53:58,994 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-05-05 15:53:58,995 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:58,995 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-05-05 15:53:58,995 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-05-05 15:53:58,995 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,995 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,996 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:58,996 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:58,996 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:58,996 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:58,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:58,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:58,997 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:58,997 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:53:58,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:58,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:58,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:58,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,997 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,998 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,998 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:58,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:58,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,998 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-05 15:53:58,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:58,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:58,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:58,998 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-05 15:53:58,998 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-05 15:53:58,998 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-05 15:53:58,998 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-05 15:53:58,998 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-05 15:53:58,998 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-05 15:53:58,998 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:58,998 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-05 15:53:58,998 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-05 15:53:58,998 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-05 15:53:58,999 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-05 15:53:58,999 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-05 15:53:58,999 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:58,998 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:53:58,999 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:53:58,999 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-05 15:53:58,999 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:58,999 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-05 15:53:58,999 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:53:58,999 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-05 15:53:58,999 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-05 15:53:58,999 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:53:58,999 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:53:58,999 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:53:58,999 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. started... 2023-05-05 15:53:58,999 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b299eb879fd7ed0f63e4b91ba5268da3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:53:59,000 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,003 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,003 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,003 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:59,003 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,003 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:59,018 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf1/8639cc4946f44be3a5974d15ad612054 2023-05-05 15:53:59,035 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf2/4d2917f1d2514972a7d495e162a2584d 2023-05-05 15:53:59,040 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf1/8639cc4946f44be3a5974d15ad612054 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/8639cc4946f44be3a5974d15ad612054 2023-05-05 15:53:59,045 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/8639cc4946f44be3a5974d15ad612054, entries=10, sequenceid=45, filesize=5.0 K 2023-05-05 15:53:59,046 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf2/4d2917f1d2514972a7d495e162a2584d as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/4d2917f1d2514972a7d495e162a2584d 2023-05-05 15:53:59,051 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/4d2917f1d2514972a7d495e162a2584d, entries=10, sequenceid=45, filesize=5.0 K 2023-05-05 15:53:59,052 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b299eb879fd7ed0f63e4b91ba5268da3 in 52ms, sequenceid=45, compaction requested=true 2023-05-05 15:53:59,052 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b299eb879fd7ed0f63e4b91ba5268da3: 2023-05-05 15:53:59,052 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:53:59,052 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:53:59,052 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:53:59,052 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,052 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-05 15:53:59,052 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-05 15:53:59,053 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,053 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:59,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:59,054 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,054 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:59,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:59,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:59,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:59,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:59,056 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-05-05 15:53:59,056 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@51d6a596[Count = 0] remaining members to acquire global barrier 2023-05-05 15:53:59,056 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-05-05 15:53:59,056 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,057 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,057 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,057 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,057 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,057 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:53:59,057 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:53:59,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,057 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-05 15:53:59,057 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-05 15:53:59,057 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:53:59,057 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:53:59,057 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:53:59,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,057 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:53:59,058 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-05 15:53:59,058 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:53:59,059 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-05 15:53:59,059 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:59,059 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-05 15:53:59,061 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,061 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-05 15:53:59,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:59,061 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-05 15:53:59,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:59,061 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:59,061 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:59,061 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-05 15:53:59,061 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-05 15:53:59,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:59,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:59,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:59,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:59,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:59,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,42799,1683302022977': 2023-05-05 15:53:59,064 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-05 15:53:59,064 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-05-05 15:53:59,064 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:53:59,064 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:53:59,064 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-05-05 15:53:59,064 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:53:59,066 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,066 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,066 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:59,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:59,066 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,066 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:59,066 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:59,066 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:59,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:59,066 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:59,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:59,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:59,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:59,066 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:59,067 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:59,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:59,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:59,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:59,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,069 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:59,069 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:59,069 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,071 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,071 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:59,071 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:59,071 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:59,071 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:59,071 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:59,071 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:59,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:59,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:59,072 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:59,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:59,072 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,072 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-05-05 15:53:59,072 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:53:59,072 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:59,072 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:59,072 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:59,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:59,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:59,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:59,072 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,072 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-05 15:53:59,072 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:59,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:59,072 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:53:59,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:59,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:59,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:59,073 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,073 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,073 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,073 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,173 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:53:59,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:53:59,189 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:53:59,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-05 15:53:59,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-05-05 15:53:59,191 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-05-05 15:53:59,191 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:59,191 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-05-05 15:53:59,192 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-05-05 15:53:59,192 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,192 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:59,193 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,193 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:59,193 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:59,193 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:59,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:59,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:59,194 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:59,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:59,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:59,194 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:53:59,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:59,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:59,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:59,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:59,195 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,195 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,195 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:59,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:59,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:59,195 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-05 15:53:59,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:59,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:59,195 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-05 15:53:59,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:59,195 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-05 15:53:59,195 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-05 15:53:59,196 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-05 15:53:59,196 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-05 15:53:59,196 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-05 15:53:59,196 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-05 15:53:59,196 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-05 15:53:59,196 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:59,196 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-05 15:53:59,196 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:59,196 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:53:59,196 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-05 15:53:59,196 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:53:59,196 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-05 15:53:59,196 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-05 15:53:59,196 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-05 15:53:59,196 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:59,196 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:53:59,196 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-05 15:53:59,196 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-05 15:53:59,196 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:53:59,197 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:53:59,197 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:53:59,197 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. started... 2023-05-05 15:53:59,197 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b299eb879fd7ed0f63e4b91ba5268da3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:53:59,198 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,199 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,199 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,199 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:59,199 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,199 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:59,207 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf1/bff65cd94e8144b7ad8e4973e9786c62 2023-05-05 15:53:59,220 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf2/9c1ddf63621a4929801ae3ae5ecd96f6 2023-05-05 15:53:59,225 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf1/bff65cd94e8144b7ad8e4973e9786c62 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/bff65cd94e8144b7ad8e4973e9786c62 2023-05-05 15:53:59,229 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/bff65cd94e8144b7ad8e4973e9786c62, entries=10, sequenceid=58, filesize=5.0 K 2023-05-05 15:53:59,230 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf2/9c1ddf63621a4929801ae3ae5ecd96f6 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/9c1ddf63621a4929801ae3ae5ecd96f6 2023-05-05 15:53:59,234 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/9c1ddf63621a4929801ae3ae5ecd96f6, entries=10, sequenceid=58, filesize=5.0 K 2023-05-05 15:53:59,235 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b299eb879fd7ed0f63e4b91ba5268da3 in 38ms, sequenceid=58, compaction requested=true 2023-05-05 15:53:59,235 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b299eb879fd7ed0f63e4b91ba5268da3: 2023-05-05 15:53:59,235 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:53:59,235 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:53:59,235 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:53:59,235 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,235 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-05 15:53:59,235 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-05 15:53:59,237 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,237 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:59,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:59,237 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,237 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:59,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:59,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:59,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:59,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:59,239 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-05-05 15:53:59,239 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5f75488f[Count = 0] remaining members to acquire global barrier 2023-05-05 15:53:59,239 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-05-05 15:53:59,239 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,240 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,240 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,240 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,240 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:53:59,240 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,240 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:53:59,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,240 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-05 15:53:59,240 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:53:59,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,240 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-05 15:53:59,240 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:53:59,241 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:53:59,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,241 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:53:59,241 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-05 15:53:59,241 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:53:59,250 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-05 15:53:59,250 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,250 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:59,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:59,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:59,250 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-05 15:53:59,250 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-05 15:53:59,250 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:59,250 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-05 15:53:59,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:59,250 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-05 15:53:59,250 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:59,250 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-05 15:53:59,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:59,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:59,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:59,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:59,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,42799,1683302022977': 2023-05-05 15:53:59,254 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-05 15:53:59,254 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-05-05 15:53:59,254 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:53:59,254 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:53:59,254 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-05-05 15:53:59,254 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:53:59,255 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,255 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,255 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,255 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,255 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:59,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,255 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:59,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,255 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:59,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:59,256 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:59,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:59,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,256 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:59,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:59,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:59,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:59,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:59,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:59,257 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:59,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:59,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:59,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,258 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,259 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:59,259 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:59,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,262 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,262 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:59,262 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:59,262 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:59,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:59,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:59,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:59,263 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:59,263 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:59,262 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,263 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-05-05 15:53:59,263 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:53:59,263 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,263 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:59,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:59,263 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:59,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:59,263 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:59,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:59,263 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,263 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:59,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:59,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:59,263 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,263 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-05 15:53:59,263 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:59,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:59,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,264 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:53:59,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:59,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:59,364 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:53:59,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:53:59,381 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:53:59,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-05 15:53:59,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-05-05 15:53:59,383 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-05-05 15:53:59,383 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:59,383 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-05-05 15:53:59,383 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-05-05 15:53:59,384 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,384 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:59,385 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:59,385 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:59,385 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:59,385 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:59,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:59,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:59,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:59,386 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,386 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:53:59,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:59,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:59,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:59,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:59,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:59,386 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,386 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,387 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:59,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:59,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:59,387 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-05 15:53:59,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:53:59,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:59,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:59,387 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-05 15:53:59,387 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-05 15:53:59,387 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-05 15:53:59,387 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-05 15:53:59,387 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-05 15:53:59,387 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:59,387 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-05 15:53:59,387 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:53:59,387 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-05 15:53:59,388 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-05 15:53:59,387 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-05 15:53:59,388 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-05 15:53:59,388 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-05 15:53:59,388 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-05 15:53:59,388 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:59,388 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-05 15:53:59,388 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:53:59,388 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:53:59,388 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:53:59,388 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-05 15:53:59,388 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-05 15:53:59,388 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:53:59,388 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:53:59,388 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:53:59,388 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. started... 2023-05-05 15:53:59,388 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b299eb879fd7ed0f63e4b91ba5268da3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:53:59,389 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,390 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,390 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,390 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:59,391 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,391 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:59,402 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf1/5a5ecf258e234499923e5da1b21d9ae5 2023-05-05 15:53:59,416 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf2/a9049db5dc314b64acc5536e55dc3800 2023-05-05 15:53:59,421 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf1/5a5ecf258e234499923e5da1b21d9ae5 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/5a5ecf258e234499923e5da1b21d9ae5 2023-05-05 15:53:59,426 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/5a5ecf258e234499923e5da1b21d9ae5, entries=10, sequenceid=71, filesize=5.0 K 2023-05-05 15:53:59,427 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf2/a9049db5dc314b64acc5536e55dc3800 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/a9049db5dc314b64acc5536e55dc3800 2023-05-05 15:53:59,432 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/a9049db5dc314b64acc5536e55dc3800, entries=10, sequenceid=71, filesize=5.0 K 2023-05-05 15:53:59,433 INFO [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b299eb879fd7ed0f63e4b91ba5268da3 in 45ms, sequenceid=71, compaction requested=true 2023-05-05 15:53:59,433 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b299eb879fd7ed0f63e4b91ba5268da3: 2023-05-05 15:53:59,433 DEBUG [rs(jenkins-hbase4.apache.org,42799,1683302022977)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:53:59,433 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:53:59,433 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:53:59,433 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,433 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-05 15:53:59,433 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-05 15:53:59,436 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,436 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:59,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:59,436 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,436 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:53:59,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:59,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:59,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:59,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:59,438 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-05-05 15:53:59,438 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4e590e01[Count = 0] remaining members to acquire global barrier 2023-05-05 15:53:59,438 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-05-05 15:53:59,438 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,439 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,439 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,439 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,440 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,440 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:53:59,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,440 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-05 15:53:59,440 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:53:59,440 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:53:59,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,440 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:53:59,440 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-05 15:53:59,440 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:53:59,440 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-05 15:53:59,440 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:53:59,440 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:53:59,441 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:59,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:59,441 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-05 15:53:59,441 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:59,441 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-05 15:53:59,442 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-05 15:53:59,442 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:59,442 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-05 15:53:59,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:59,444 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-05 15:53:59,444 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:59,444 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-05 15:53:59,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:59,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:59,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:59,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:59,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,42799,1683302022977': 2023-05-05 15:53:59,447 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-05 15:53:59,447 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-05-05 15:53:59,447 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:53:59,447 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:53:59,447 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-05-05 15:53:59,447 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:53:59,449 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,449 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,449 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,449 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:59,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:53:59,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:53:59,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,449 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,449 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:59,449 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:59,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:59,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:53:59,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:59,450 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:59,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:59,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:59,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:59,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,450 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:59,451 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:53:59,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:59,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,453 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:53:59,453 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-05 15:53:59,453 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,458 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,458 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:59,458 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:59,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:59,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:59,458 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:59,458 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:53:59,458 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:59,458 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-05-05 15:53:59,458 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:53:59,458 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:53:59,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:59,458 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:53:59,458 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:59,458 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:53:59,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:59,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:53:59,459 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-05 15:53:59,458 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,459 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:53:59,459 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-05 15:53:59,459 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:53:59,459 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,459 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:53:59,459 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,459 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-05 15:53:59,459 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-05 15:53:59,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:59,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:59,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:59,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:53:59,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:59,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:53:59,559 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:53:59,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:53:59,821 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-05-05 15:53:59,822 DEBUG [Listener at localhost/35117] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:53:59,823 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:53:59,824 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:53:59,824 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:53:59,824 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:53:59,825 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:53:59,825 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:53:59,825 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:53:59,825 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:54:00,073 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,074 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:54:00,074 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:54:00,074 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:54:00,074 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-05-05 15:54:00,074 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:54:00,074 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:54:00,074 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:54:00,074 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:54:00,075 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:54:00,076 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-05-05 15:54:00,076 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:54:00,077 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:54:00,077 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:54:00,077 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:54:00,078 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:00,078 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:54:00,078 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:00,078 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:00,079 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:00,079 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:54:00,079 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:00,079 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:00,079 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:00,080 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,080 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:54:00,080 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,080 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,080 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,080 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-05 15:54:00,081 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:54:00,081 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:54:00,082 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302070082 2023-05-05 15:54:00,083 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:54:00,083 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,085 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:54:00,086 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=b299eb879fd7ed0f63e4b91ba5268da3, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42799,1683302022977, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977, openSeqNum=2 2023-05-05 15:54:00,086 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:54:00,086 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302100086 2023-05-05 15:54:00,086 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302160086 2023-05-05 15:54:00,086 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-05 15:54:00,086 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:54:00,088 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:54:00,088 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,088 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:54:00,088 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:54:00,336 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,336 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:54:00,336 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:54:00,336 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:54:00,339 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:54:00,339 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:54:00,339 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:54:00,339 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:54:00,339 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:54:00,340 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:54:00,341 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-05-05 15:54:00,342 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:54:00,342 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:54:00,342 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:54:00,342 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:54:00,343 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:00,343 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:54:00,343 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:00,343 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:00,344 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:00,344 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:54:00,344 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:00,345 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:00,345 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:00,345 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,346 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:54:00,346 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,346 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,346 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,346 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-05 15:54:00,347 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:54:00,347 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:54:00,348 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302070348 2023-05-05 15:54:00,349 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,349 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:54:00,351 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:54:00,352 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=b299eb879fd7ed0f63e4b91ba5268da3, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42799,1683302022977, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977, openSeqNum=2 2023-05-05 15:54:00,352 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:54:00,352 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302100352 2023-05-05 15:54:00,352 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302160352 2023-05-05 15:54:00,352 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-05 15:54:00,352 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:54:00,354 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN}] 2023-05-05 15:54:00,354 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:54:00,354 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,354 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:54:00,354 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:54:00,602 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,602 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:54:00,602 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:54:00,602 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:54:00,603 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:54:00,603 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:54:00,603 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:54:00,603 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:54:00,603 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:54:00,604 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:54:00,605 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-05-05 15:54:00,605 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:54:00,605 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:54:00,605 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:54:00,606 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:54:00,606 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:00,606 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:54:00,606 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:00,606 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:00,607 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:00,607 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:54:00,607 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:00,608 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:00,608 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:00,609 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,609 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:54:00,609 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,609 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,609 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,609 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-05 15:54:00,610 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:54:00,610 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:54:00,610 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302070610 2023-05-05 15:54:00,612 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:54:00,612 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,614 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:54:00,615 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=b299eb879fd7ed0f63e4b91ba5268da3, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42799,1683302022977, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977, openSeqNum=2 2023-05-05 15:54:00,615 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:54:00,615 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302100615 2023-05-05 15:54:00,615 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302160615 2023-05-05 15:54:00,615 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-05 15:54:00,615 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN}] 2023-05-05 15:54:00,615 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:54:00,617 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN 2023-05-05 15:54:00,618 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=b299eb879fd7ed0f63e4b91ba5268da3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:00,618 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683302040618"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302040618"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302040618"}]},"ts":"1683302040618"} 2023-05-05 15:54:00,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977}] 2023-05-05 15:54:00,620 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:54:00,620 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN 2023-05-05 15:54:00,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:54:00,620 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:54:00,865 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,865 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:54:00,865 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:54:00,865 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:54:00,866 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:54:00,867 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:54:00,867 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:54:00,867 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:54:00,867 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:54:00,868 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:54:00,868 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-05 15:54:00,869 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-05-05 15:54:00,869 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:54:00,869 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:54:00,869 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:54:00,869 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:54:00,870 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:00,870 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:54:00,870 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:00,870 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:00,871 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:00,871 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:54:00,871 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:00,871 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:00,871 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:00,872 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,872 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN 2023-05-05 15:54:00,872 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:54:00,872 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,872 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,873 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:00,873 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:54:00,873 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN 2023-05-05 15:54:00,873 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-05 15:54:00,873 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:54:00,874 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:54:00,874 INFO [Listener at localhost/35117] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3 to restore RIT 2023-05-05 15:54:00,876 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302070876 2023-05-05 15:54:00,877 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN 2023-05-05 15:54:00,877 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:54:00,879 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:54:00,879 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=b299eb879fd7ed0f63e4b91ba5268da3, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,42799,1683302022977, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977, openSeqNum=2 2023-05-05 15:54:00,879 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:54:00,879 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302100879 2023-05-05 15:54:00,879 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302160879 2023-05-05 15:54:00,879 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-05 15:54:00,880 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=b299eb879fd7ed0f63e4b91ba5268da3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:00,880 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683302040880"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302040880"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302040880"}]},"ts":"1683302040880"} 2023-05-05 15:54:00,881 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977}] 2023-05-05 15:54:00,881 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:54:01,034 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,034 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b299eb879fd7ed0f63e4b91ba5268da3, disabling compactions & flushes 2023-05-05 15:54:01,034 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:54:01,034 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:54:01,034 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. after waiting 0 ms 2023-05-05 15:54:01,034 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:54:01,035 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing b299eb879fd7ed0f63e4b91ba5268da3 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-05 15:54:01,045 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf1/7aacee3dad2e4f21a38483dab3b0197d 2023-05-05 15:54:01,057 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf2/4441f0039a044a6b9af4232fd7b84326 2023-05-05 15:54:01,062 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf1/7aacee3dad2e4f21a38483dab3b0197d as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/7aacee3dad2e4f21a38483dab3b0197d 2023-05-05 15:54:01,066 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/7aacee3dad2e4f21a38483dab3b0197d, entries=8, sequenceid=82, filesize=5.0 K 2023-05-05 15:54:01,067 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf2/4441f0039a044a6b9af4232fd7b84326 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/4441f0039a044a6b9af4232fd7b84326 2023-05-05 15:54:01,072 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/4441f0039a044a6b9af4232fd7b84326, entries=8, sequenceid=82, filesize=5.0 K 2023-05-05 15:54:01,073 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for b299eb879fd7ed0f63e4b91ba5268da3 in 37ms, sequenceid=82, compaction requested=true 2023-05-05 15:54:01,078 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-05 15:54:01,078 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:54:01,078 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:54:01,079 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b299eb879fd7ed0f63e4b91ba5268da3: 2023-05-05 15:54:01,081 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,081 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=b299eb879fd7ed0f63e4b91ba5268da3, regionState=CLOSED 2023-05-05 15:54:01,082 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683302041081"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302041081"}]},"ts":"1683302041081"} 2023-05-05 15:54:01,083 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:54:01,083 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:01,083 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:54:01,083 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:54:01,130 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:01,130 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:54:01,130 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:54:01,130 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:54:01,131 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:54:01,131 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:54:01,131 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:54:01,131 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:54:01,131 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:54:01,132 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:54:01,133 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-05 15:54:01,133 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-05-05 15:54:01,133 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-05-05 15:54:01,133 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:54:01,133 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:54:01,134 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:54:01,134 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:54:01,134 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:01,135 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:54:01,135 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:01,135 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:01,135 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:01,136 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:54:01,136 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:01,136 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:01,136 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:01,137 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:01,137 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN 2023-05-05 15:54:01,137 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:01,137 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:54:01,137 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:01,137 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:01,137 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:01,138 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:54:01,138 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN 2023-05-05 15:54:01,138 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:54:01,138 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-05 15:54:01,139 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:54:01,139 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:54:01,139 INFO [Listener at localhost/35117] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3 to restore RIT 2023-05-05 15:54:01,140 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302071140 2023-05-05 15:54:01,142 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:54:01,142 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:01,144 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:54:01,145 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=b299eb879fd7ed0f63e4b91ba5268da3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42799,1683302022977, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977, openSeqNum=2 2023-05-05 15:54:01,145 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:54:01,145 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302101145 2023-05-05 15:54:01,145 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302161145 2023-05-05 15:54:01,145 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-05 15:54:01,145 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=b299eb879fd7ed0f63e4b91ba5268da3, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:01,145 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683302041145"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302041145"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302041145"}]},"ts":"1683302041145"} 2023-05-05 15:54:01,147 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:54:01,150 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-05-05 15:54:01,151 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 in 266 msec 2023-05-05 15:54:01,151 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:54:01,151 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN 2023-05-05 15:54:01,151 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:54:01,151 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:54:01,395 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:01,395 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:54:01,395 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:54:01,396 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:54:01,396 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:54:01,396 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:54:01,396 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:54:01,396 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:54:01,396 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:54:01,397 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:54:01,398 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-05 15:54:01,398 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-05-05 15:54:01,398 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-05 15:54:01,398 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:54:01,399 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:54:01,399 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:54:01,399 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:54:01,399 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:01,400 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:54:01,400 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:01,400 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:01,400 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:01,401 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:54:01,401 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:01,401 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:01,401 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:01,402 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:01,402 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN 2023-05-05 15:54:01,402 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:01,402 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:54:01,402 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:01,403 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:01,403 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:01,403 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:54:01,403 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:54:01,403 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-05 15:54:01,403 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:54:01,404 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:54:01,404 INFO [Listener at localhost/35117] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3 to restore RIT 2023-05-05 15:54:01,404 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302071404 2023-05-05 15:54:01,405 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN 2023-05-05 15:54:01,405 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:54:01,407 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:54:01,408 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=b299eb879fd7ed0f63e4b91ba5268da3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42799,1683302022977, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977, openSeqNum=2 2023-05-05 15:54:01,408 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:54:01,408 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302101408 2023-05-05 15:54:01,408 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302161408 2023-05-05 15:54:01,408 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-05 15:54:01,408 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN 2023-05-05 15:54:01,410 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:54:01,412 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-05-05 15:54:01,413 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN in 795 msec 2023-05-05 15:54:01,415 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:54:01,415 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:01,415 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:54:01,415 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:54:01,658 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:01,658 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:54:01,658 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:54:01,659 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:54:01,659 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:54:01,659 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:54:01,659 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:54:01,659 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:54:01,659 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:54:01,660 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:54:01,661 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-05 15:54:01,661 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-05 15:54:01,661 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-05 15:54:01,661 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:54:01,661 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:54:01,661 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:54:01,662 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:54:01,662 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:01,662 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:54:01,663 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:01,663 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:01,663 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:01,664 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:54:01,664 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:01,664 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:01,664 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:01,665 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:01,665 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN 2023-05-05 15:54:01,665 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:01,665 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:54:01,665 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:54:01,665 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:54:01,665 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-05 15:54:01,666 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:54:01,666 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:54:01,667 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302071666 2023-05-05 15:54:01,667 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:54:01,668 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:01,670 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:54:01,670 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=b299eb879fd7ed0f63e4b91ba5268da3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42799,1683302022977, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977, openSeqNum=2 2023-05-05 15:54:01,670 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:54:01,670 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302101670 2023-05-05 15:54:01,670 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302161670 2023-05-05 15:54:01,671 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-05 15:54:01,671 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:01,671 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:01,671 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:01,675 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:54:01,684 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=b299eb879fd7ed0f63e4b91ba5268da3, threads=14 2023-05-05 15:54:01,685 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/1ea30cc7aa0f49b4bb29289547c266bb for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,685 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/4441f0039a044a6b9af4232fd7b84326 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,685 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/4d2917f1d2514972a7d495e162a2584d for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,685 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/8502d9b6ae874806a9213be374c9565d for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,685 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/9c1ddf63621a4929801ae3ae5ecd96f6 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,685 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/a9049db5dc314b64acc5536e55dc3800 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,686 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/fd1f211bab184c0a9df23ab9a2232ea2 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,686 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/5a5ecf258e234499923e5da1b21d9ae5 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,686 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/7aacee3dad2e4f21a38483dab3b0197d for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,686 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/8639cc4946f44be3a5974d15ad612054 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,686 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/9ce680e0fa604098a7aa8ac4b9b7166c for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,687 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/9eac5d62b79f49159581b82e191d6f4a for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,687 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/a79dfb5adfe544bd973d9313c282fdcb for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,687 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/bff65cd94e8144b7ad8e4973e9786c62 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,825 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/5a5ecf258e234499923e5da1b21d9ae5 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,855 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/a9049db5dc314b64acc5536e55dc3800 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,856 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/4441f0039a044a6b9af4232fd7b84326 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,857 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-05 15:54:01,925 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/4d2917f1d2514972a7d495e162a2584d for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,940 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/9ce680e0fa604098a7aa8ac4b9b7166c for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,950 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/1ea30cc7aa0f49b4bb29289547c266bb for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,950 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/bff65cd94e8144b7ad8e4973e9786c62 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,960 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/9eac5d62b79f49159581b82e191d6f4a for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,964 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/a79dfb5adfe544bd973d9313c282fdcb for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,967 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/9c1ddf63621a4929801ae3ae5ecd96f6 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,984 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/fd1f211bab184c0a9df23ab9a2232ea2 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:01,992 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/8502d9b6ae874806a9213be374c9565d for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,194 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/7aacee3dad2e4f21a38483dab3b0197d for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,248 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/8639cc4946f44be3a5974d15ad612054 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,249 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region b299eb879fd7ed0f63e4b91ba5268da3 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-05 15:54:02,289 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:54:02,289 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:02,289 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:54:02,289 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:54:02,421 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:02,421 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:54:02,421 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:54:02,421 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:54:02,421 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:54:02,421 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:54:02,422 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:54:02,422 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:54:02,422 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:54:02,423 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:54:02,423 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-05-05 15:54:02,424 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-05 15:54:02,424 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-05 15:54:02,424 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:54:02,425 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:54:02,425 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:54:02,425 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:54:02,425 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:02,426 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:54:02,426 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:02,426 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:02,426 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:02,426 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:54:02,427 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:02,427 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:02,427 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:02,427 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:02,428 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN 2023-05-05 15:54:02,428 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:02,428 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:54:02,428 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:02,428 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:02,428 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:02,428 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:54:02,428 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:54:02,428 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-05 15:54:02,429 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:54:02,429 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:54:02,430 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302072430 2023-05-05 15:54:02,431 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:54:02,432 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:02,434 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:54:02,434 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=b299eb879fd7ed0f63e4b91ba5268da3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42799,1683302022977, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977, openSeqNum=2 2023-05-05 15:54:02,434 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:54:02,434 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302102434 2023-05-05 15:54:02,434 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302162434 2023-05-05 15:54:02,434 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-05 15:54:02,443 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=b299eb879fd7ed0f63e4b91ba5268da3, threads=14 2023-05-05 15:54:02,444 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/1ea30cc7aa0f49b4bb29289547c266bb for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,444 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/4441f0039a044a6b9af4232fd7b84326 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,444 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/4d2917f1d2514972a7d495e162a2584d for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,444 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/8502d9b6ae874806a9213be374c9565d for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,445 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/9c1ddf63621a4929801ae3ae5ecd96f6 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,445 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/a9049db5dc314b64acc5536e55dc3800 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,445 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/fd1f211bab184c0a9df23ab9a2232ea2 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,445 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/5a5ecf258e234499923e5da1b21d9ae5 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,445 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/7aacee3dad2e4f21a38483dab3b0197d for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,445 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/8639cc4946f44be3a5974d15ad612054 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,445 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/9ce680e0fa604098a7aa8ac4b9b7166c for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,445 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/9eac5d62b79f49159581b82e191d6f4a for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,446 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/a79dfb5adfe544bd973d9313c282fdcb for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,446 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/bff65cd94e8144b7ad8e4973e9786c62 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,606 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/9ce680e0fa604098a7aa8ac4b9b7166c for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,627 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/a9049db5dc314b64acc5536e55dc3800 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,629 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/fd1f211bab184c0a9df23ab9a2232ea2 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,631 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/4d2917f1d2514972a7d495e162a2584d for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,633 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/9eac5d62b79f49159581b82e191d6f4a for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,653 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/9c1ddf63621a4929801ae3ae5ecd96f6 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,654 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/8639cc4946f44be3a5974d15ad612054 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,655 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/1ea30cc7aa0f49b4bb29289547c266bb for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,655 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/8502d9b6ae874806a9213be374c9565d for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,657 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/bff65cd94e8144b7ad8e4973e9786c62 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,658 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/5a5ecf258e234499923e5da1b21d9ae5 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,658 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/4441f0039a044a6b9af4232fd7b84326 for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:02,661 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/a79dfb5adfe544bd973d9313c282fdcb for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:03,013 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/7aacee3dad2e4f21a38483dab3b0197d for region: b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:03,013 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region b299eb879fd7ed0f63e4b91ba5268da3 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-05 15:54:03,054 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:54:03,059 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b13dfdeca1ae2cc48d85f970e4c58cc2/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-05 15:54:03,061 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/e73cfd57ec9ec904b48f85318b83d6ac/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-05 15:54:03,061 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:54:03,061 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,061 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:54:03,061 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:54:03,184 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,185 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:54:03,185 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:54:03,185 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:54:03,186 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:54:03,186 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:54:03,186 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:54:03,186 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:54:03,186 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:54:03,187 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:54:03,188 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-05-05 15:54:03,188 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-05 15:54:03,188 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-05 15:54:03,188 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:54:03,189 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:54:03,189 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:54:03,189 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:54:03,190 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:03,190 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:54:03,190 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:03,190 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:03,191 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:03,191 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:54:03,191 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:03,192 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:03,192 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:03,192 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,193 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN 2023-05-05 15:54:03,193 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:03,195 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:54:03,195 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,195 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,195 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,195 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:54:03,195 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:54:03,196 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-05-05 15:54:03,196 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:54:03,197 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:54:03,197 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302073197 2023-05-05 15:54:03,198 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,198 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:54:03,200 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:54:03,201 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=b299eb879fd7ed0f63e4b91ba5268da3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42799,1683302022977, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977, openSeqNum=2 2023-05-05 15:54:03,201 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:54:03,201 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302103201 2023-05-05 15:54:03,201 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302163201 2023-05-05 15:54:03,201 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-05 15:54:03,204 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:54:03,207 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:54:03,207 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,207 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:54:03,207 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:54:03,452 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,452 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:54:03,452 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:54:03,452 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:54:03,452 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:54:03,452 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:54:03,452 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:54:03,452 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:54:03,452 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:54:03,453 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:54:03,454 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-05-05 15:54:03,454 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-05 15:54:03,454 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-05 15:54:03,455 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:54:03,455 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:54:03,455 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:54:03,456 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:54:03,456 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:03,456 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:54:03,457 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:03,457 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:03,457 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:03,458 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:54:03,458 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:03,458 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:03,459 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:03,459 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,459 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN 2023-05-05 15:54:03,459 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:03,460 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:54:03,460 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,460 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,460 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,460 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:54:03,460 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:54:03,460 DEBUG [Listener at localhost/35117] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,460 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-05 15:54:03,461 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:54:03,462 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:54:03,463 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302073463 2023-05-05 15:54:03,463 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:54:03,465 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,467 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:54:03,468 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=b299eb879fd7ed0f63e4b91ba5268da3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42799,1683302022977, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977, openSeqNum=2 2023-05-05 15:54:03,468 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:54:03,468 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302103468 2023-05-05 15:54:03,468 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302163468 2023-05-05 15:54:03,468 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-05-05 15:54:03,468 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:54:03,481 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:54:03,481 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-05 15:54:03,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:54:03,481 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:54:03,719 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,720 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:54:03,720 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:54:03,720 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:54:03,721 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:54:03,721 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:54:03,721 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:54:03,721 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:54:03,721 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:54:03,722 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:54:03,723 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-05-05 15:54:03,723 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-05 15:54:03,723 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-05 15:54:03,723 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:54:03,723 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:54:03,724 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:54:03,724 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:54:03,724 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:03,724 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:54:03,724 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:03,725 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:03,725 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:03,725 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:54:03,725 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:03,725 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:03,726 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:03,727 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,727 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN 2023-05-05 15:54:03,727 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:03,727 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:54:03,727 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,727 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,727 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,727 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:54:03,727 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:54:03,727 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-05 15:54:03,728 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:54:03,728 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:54:03,729 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302073729 2023-05-05 15:54:03,732 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:54:03,733 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,735 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:54:03,735 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=b299eb879fd7ed0f63e4b91ba5268da3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42799,1683302022977, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977, openSeqNum=2 2023-05-05 15:54:03,735 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:54:03,735 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302103735 2023-05-05 15:54:03,735 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302163735 2023-05-05 15:54:03,735 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-05 15:54:03,736 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:54:03,738 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:54:03,738 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-05 15:54:03,738 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:54:03,738 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:54:03,986 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,987 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:54:03,987 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:54:03,987 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:54:03,987 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:54:03,987 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:54:03,987 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:54:03,987 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:54:03,987 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:54:03,988 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:54:03,989 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-05-05 15:54:03,989 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-05 15:54:03,989 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-05 15:54:03,989 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:54:03,990 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:54:03,990 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:54:03,990 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:54:03,991 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:03,991 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:54:03,991 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:03,991 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:03,992 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:03,992 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:54:03,992 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:03,992 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:03,993 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:03,994 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,994 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN 2023-05-05 15:54:03,994 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:03,994 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:54:03,994 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,994 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,994 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:03,994 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:54:03,994 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:54:03,995 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-05-05 15:54:03,995 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:54:03,996 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:54:03,996 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302073996 2023-05-05 15:54:03,997 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:54:03,998 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:04,000 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:54:04,000 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=b299eb879fd7ed0f63e4b91ba5268da3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42799,1683302022977, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977, openSeqNum=2 2023-05-05 15:54:04,000 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:54:04,000 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302104000 2023-05-05 15:54:04,000 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302164000 2023-05-05 15:54:04,000 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-05 15:54:04,001 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:54:04,003 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:54:04,003 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-05 15:54:04,003 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:54:04,003 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:54:04,251 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:04,251 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:54:04,251 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:54:04,251 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:54:04,252 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:54:04,252 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:54:04,252 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:54:04,252 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:54:04,252 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:54:04,253 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:54:04,254 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-05-05 15:54:04,254 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-05 15:54:04,254 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-05 15:54:04,254 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:54:04,254 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:54:04,255 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:54:04,255 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:54:04,255 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:04,255 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:54:04,255 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:04,256 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:04,256 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:04,256 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:54:04,256 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:04,256 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:04,256 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:04,257 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:04,257 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN 2023-05-05 15:54:04,257 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:04,257 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:54:04,257 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:04,257 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:04,257 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:04,257 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:54:04,257 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:54:04,258 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-05 15:54:04,259 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:54:04,259 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:54:04,260 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302074260 2023-05-05 15:54:04,260 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:54:04,262 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:04,264 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:54:04,264 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=b299eb879fd7ed0f63e4b91ba5268da3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42799,1683302022977, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977, openSeqNum=2 2023-05-05 15:54:04,264 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:54:04,264 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302104264 2023-05-05 15:54:04,264 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302164264 2023-05-05 15:54:04,264 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-05 15:54:04,264 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:54:04,268 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN] 2023-05-05 15:54:04,268 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:54:04,268 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-05 15:54:04,268 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:54:04,268 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:54:04,514 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:04,515 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:54:04,515 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:54:04,515 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:54:04,517 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:54:04,517 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:54:04,517 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:54:04,517 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:54:04,517 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:54:04,518 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:54:04,519 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-05-05 15:54:04,519 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-05 15:54:04,519 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-05 15:54:04,519 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-05-05 15:54:04,519 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:54:04,520 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:54:04,520 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:54:04,520 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:54:04,520 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:04,520 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:54:04,521 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:04,521 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:04,521 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:04,521 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:54:04,521 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:04,521 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:04,522 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:04,522 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:04,523 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN 2023-05-05 15:54:04,523 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:04,523 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:54:04,523 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:54:04,523 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:04,523 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:04,523 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:04,523 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:54:04,523 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:54:04,523 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:54:04,523 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-05 15:54:04,524 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:54:04,524 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:54:04,525 INFO [Listener at localhost/35117] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3 to restore RIT 2023-05-05 15:54:04,525 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302074525 2023-05-05 15:54:04,526 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:54:04,527 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:04,528 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:54:04,529 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:54:04,530 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=b299eb879fd7ed0f63e4b91ba5268da3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42799,1683302022977, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977, openSeqNum=2 2023-05-05 15:54:04,530 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:54:04,530 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302104530 2023-05-05 15:54:04,530 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302164530 2023-05-05 15:54:04,530 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-05 15:54:04,530 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:54:04,535 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:54:04,535 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-05 15:54:04,536 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:54:04,536 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:54:04,780 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:04,780 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:54:04,780 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:54:04,780 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:54:04,781 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:54:04,781 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:54:04,781 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:54:04,781 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:54:04,781 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:54:04,781 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:54:04,782 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-05 15:54:04,782 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-05-05 15:54:04,782 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:54:04,783 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:54:04,783 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:54:04,783 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:54:04,783 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:04,784 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:54:04,784 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:04,784 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:04,784 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:04,784 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:54:04,784 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:04,784 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:04,785 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:04,785 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:04,785 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:54:04,785 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:54:04,785 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:04,785 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:04,785 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:04,785 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:54:04,786 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-05-05 15:54:04,786 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:54:04,787 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:54:04,787 INFO [Listener at localhost/35117] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3 to restore RIT 2023-05-05 15:54:04,787 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302074787 2023-05-05 15:54:04,788 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:54:04,790 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:04,790 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:54:04,792 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:54:04,792 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=b299eb879fd7ed0f63e4b91ba5268da3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42799,1683302022977, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977, openSeqNum=2 2023-05-05 15:54:04,792 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:54:04,792 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302104792 2023-05-05 15:54:04,792 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302164792 2023-05-05 15:54:04,793 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-05 15:54:04,793 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:54:04,795 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:54:04,795 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-05 15:54:04,795 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:54:04,795 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:54:05,043 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:05,043 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:54:05,043 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:54:05,043 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:54:05,044 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:54:05,044 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:54:05,044 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:54:05,044 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:54:05,044 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:54:05,045 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:54:05,046 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-05-05 15:54:05,046 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-05-05 15:54:05,046 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:54:05,047 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:54:05,047 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:54:05,047 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:54:05,048 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:05,048 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:54:05,048 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:05,048 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:05,049 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:05,049 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:54:05,049 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:05,049 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:05,050 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:05,050 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:05,050 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:54:05,051 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:54:05,051 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:05,051 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:05,051 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:05,051 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:54:05,051 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-05-05 15:54:05,052 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:54:05,052 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:54:05,053 INFO [Listener at localhost/35117] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3 to restore RIT 2023-05-05 15:54:05,053 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302075053 2023-05-05 15:54:05,055 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:05,055 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:54:05,056 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:54:05,058 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:54:05,058 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=b299eb879fd7ed0f63e4b91ba5268da3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42799,1683302022977, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977, openSeqNum=2 2023-05-05 15:54:05,058 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:54:05,058 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302105058 2023-05-05 15:54:05,058 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302165058 2023-05-05 15:54:05,058 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-05 15:54:05,059 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:54:05,061 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:54:05,061 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-05 15:54:05,061 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:54:05,061 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:54:05,309 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:05,309 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:54:05,309 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:54:05,309 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:54:05,309 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:54:05,309 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:54:05,310 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:54:05,310 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:54:05,310 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:54:05,311 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:54:05,312 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-05-05 15:54:05,312 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-05-05 15:54:05,312 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:54:05,313 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:54:05,313 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:54:05,313 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:54:05,313 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:05,314 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:54:05,314 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:05,314 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:05,314 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:05,315 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:54:05,315 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:05,315 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:05,315 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:05,316 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:05,316 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:54:05,316 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:54:05,316 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:05,316 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:05,316 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:05,316 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:54:05,317 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-05 15:54:05,317 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:54:05,318 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:54:05,318 INFO [Listener at localhost/35117] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3 to restore RIT 2023-05-05 15:54:05,318 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302075318 2023-05-05 15:54:05,319 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:54:05,320 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:05,322 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:54:05,324 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:54:05,324 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=b299eb879fd7ed0f63e4b91ba5268da3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42799,1683302022977, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977, openSeqNum=2 2023-05-05 15:54:05,324 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:54:05,324 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302105324 2023-05-05 15:54:05,325 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302165325 2023-05-05 15:54:05,325 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-05-05 15:54:05,325 INFO [PEWorker-1] procedure.MasterProcedureScheduler(718): Waiting on xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN held by pid=44 2023-05-05 15:54:05,325 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:54:05,325 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:54:05,327 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac exec-time=5.5030 sec 2023-05-05 15:54:05,327 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:54:05,328 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,42799,1683302022977; forceNewPlan=false, retain=true 2023-05-05 15:54:05,478 INFO [jenkins-hbase4:44213] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-05 15:54:05,480 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=b299eb879fd7ed0f63e4b91ba5268da3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:05,480 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683302045480"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302045480"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302045480"}]},"ts":"1683302045480"} 2023-05-05 15:54:05,482 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977}] 2023-05-05 15:54:05,482 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:54:05,482 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:54:05,482 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:54:05,482 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:54:05,575 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-05 15:54:05,575 WARN [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-05-05 15:54:05,575 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:54:05,575 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:54:05,575 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:54:05,575 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:54:05,575 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:54:05,576 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:54:05,576 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:54:05,576 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:54:05,577 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:54:05,577 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-05-05 15:54:05,578 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:54:05,578 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:54:05,578 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:54:05,578 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:54:05,578 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:05,579 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:54:05,579 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:05,579 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:54:05,579 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:05,579 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:54:05,579 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:05,579 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:54:05,580 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:05,580 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:54:05,580 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:54:05,580 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:54:05,580 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:54:05,580 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-05-05 15:54:05,581 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:54:05,584 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:54:05,584 INFO [Listener at localhost/35117] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3 to restore RIT 2023-05-05 15:54:05,586 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302075586 2023-05-05 15:54:05,587 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:54:05,587 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:54:05,590 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:54:05,590 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=b299eb879fd7ed0f63e4b91ba5268da3, regionState=OPENING, lastHost=jenkins-hbase4.apache.org,42799,1683302022977, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977, openSeqNum=2 2023-05-05 15:54:05,590 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:54:05,590 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302105590 2023-05-05 15:54:05,590 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302165590 2023-05-05 15:54:05,590 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-05 15:54:05,590 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=b299eb879fd7ed0f63e4b91ba5268da3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:05,591 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683302045590"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302045590"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302045590"}]},"ts":"1683302045590"} 2023-05-05 15:54:05,592 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977}] 2023-05-05 15:54:05,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:54:05,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b299eb879fd7ed0f63e4b91ba5268da3, NAME => 'testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.', STARTKEY => '', ENDKEY => ''} 2023-05-05 15:54:05,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:54:05,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:05,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:54:05,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:05,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:05,750 INFO [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:05,751 DEBUG [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1 2023-05-05 15:54:05,751 DEBUG [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1 2023-05-05 15:54:05,752 INFO [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b299eb879fd7ed0f63e4b91ba5268da3 columnFamilyName cf1 2023-05-05 15:54:05,759 DEBUG [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/5a5ecf258e234499923e5da1b21d9ae5 2023-05-05 15:54:05,764 DEBUG [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/7aacee3dad2e4f21a38483dab3b0197d 2023-05-05 15:54:05,767 DEBUG [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/8639cc4946f44be3a5974d15ad612054 2023-05-05 15:54:05,770 DEBUG [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/9ce680e0fa604098a7aa8ac4b9b7166c 2023-05-05 15:54:05,778 DEBUG [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/9eac5d62b79f49159581b82e191d6f4a 2023-05-05 15:54:05,782 DEBUG [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/a79dfb5adfe544bd973d9313c282fdcb 2023-05-05 15:54:05,785 DEBUG [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/bff65cd94e8144b7ad8e4973e9786c62 2023-05-05 15:54:05,785 INFO [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] regionserver.HStore(310): Store=b299eb879fd7ed0f63e4b91ba5268da3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:54:05,786 INFO [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:05,787 DEBUG [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2 2023-05-05 15:54:05,787 DEBUG [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2 2023-05-05 15:54:05,787 INFO [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b299eb879fd7ed0f63e4b91ba5268da3 columnFamilyName cf2 2023-05-05 15:54:05,793 DEBUG [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/1ea30cc7aa0f49b4bb29289547c266bb 2023-05-05 15:54:05,797 DEBUG [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/4441f0039a044a6b9af4232fd7b84326 2023-05-05 15:54:05,800 DEBUG [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/4d2917f1d2514972a7d495e162a2584d 2023-05-05 15:54:05,804 DEBUG [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/8502d9b6ae874806a9213be374c9565d 2023-05-05 15:54:05,807 DEBUG [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/9c1ddf63621a4929801ae3ae5ecd96f6 2023-05-05 15:54:05,811 DEBUG [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/a9049db5dc314b64acc5536e55dc3800 2023-05-05 15:54:05,814 DEBUG [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/fd1f211bab184c0a9df23ab9a2232ea2 2023-05-05 15:54:05,814 INFO [StoreOpener-b299eb879fd7ed0f63e4b91ba5268da3-1] regionserver.HStore(310): Store=b299eb879fd7ed0f63e4b91ba5268da3/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:54:05,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:05,815 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:05,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.splits 2023-05-05 15:54:05,818 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-05 15:54:05,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:05,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b299eb879fd7ed0f63e4b91ba5268da3; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64934262, jitterRate=-0.0324040949344635}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-05 15:54:05,820 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b299eb879fd7ed0f63e4b91ba5268da3: 2023-05-05 15:54:05,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3., pid=48, masterSystemTime=1683302045745 2023-05-05 15:54:05,821 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:54:05,822 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-05 15:54:05,822 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-05 15:54:05,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:54:05,823 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-05-05 15:54:05,823 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-05-05 15:54:05,823 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.HStore(1912): b299eb879fd7ed0f63e4b91ba5268da3/cf1 is initiating minor compaction (all files) 2023-05-05 15:54:05,823 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.HStore(1912): b299eb879fd7ed0f63e4b91ba5268da3/cf2 is initiating minor compaction (all files) 2023-05-05 15:54:05,823 INFO [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of b299eb879fd7ed0f63e4b91ba5268da3/cf1 in testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:54:05,823 INFO [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.HRegion(2259): Starting compaction of b299eb879fd7ed0f63e4b91ba5268da3/cf2 in testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:54:05,823 INFO [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/1ea30cc7aa0f49b4bb29289547c266bb, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/8502d9b6ae874806a9213be374c9565d, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/fd1f211bab184c0a9df23ab9a2232ea2, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/4d2917f1d2514972a7d495e162a2584d, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/9c1ddf63621a4929801ae3ae5ecd96f6, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/a9049db5dc314b64acc5536e55dc3800, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/4441f0039a044a6b9af4232fd7b84326] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp, totalSize=34.9 K 2023-05-05 15:54:05,823 INFO [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/9eac5d62b79f49159581b82e191d6f4a, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/a79dfb5adfe544bd973d9313c282fdcb, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/9ce680e0fa604098a7aa8ac4b9b7166c, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/8639cc4946f44be3a5974d15ad612054, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/bff65cd94e8144b7ad8e4973e9786c62, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/5a5ecf258e234499923e5da1b21d9ae5, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/7aacee3dad2e4f21a38483dab3b0197d] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp, totalSize=34.9 K 2023-05-05 15:54:05,824 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:54:05,824 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] compactions.Compactor(207): Compacting 1ea30cc7aa0f49b4bb29289547c266bb, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683302038381 2023-05-05 15:54:05,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:54:05,825 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] compactions.Compactor(207): Compacting 9eac5d62b79f49159581b82e191d6f4a, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683302038381 2023-05-05 15:54:05,825 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=b299eb879fd7ed0f63e4b91ba5268da3, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:05,825 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] compactions.Compactor(207): Compacting 8502d9b6ae874806a9213be374c9565d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683302038570 2023-05-05 15:54:05,825 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] compactions.Compactor(207): Compacting a79dfb5adfe544bd973d9313c282fdcb, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683302038570 2023-05-05 15:54:05,825 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683302045825"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302045825"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302045825"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302045825"}]},"ts":"1683302045825"} 2023-05-05 15:54:05,825 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] compactions.Compactor(207): Compacting fd1f211bab184c0a9df23ab9a2232ea2, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683302038779 2023-05-05 15:54:05,826 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] compactions.Compactor(207): Compacting 9ce680e0fa604098a7aa8ac4b9b7166c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683302038779 2023-05-05 15:54:05,826 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] compactions.Compactor(207): Compacting 4d2917f1d2514972a7d495e162a2584d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683302038979 2023-05-05 15:54:05,826 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] compactions.Compactor(207): Compacting 8639cc4946f44be3a5974d15ad612054, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683302038979 2023-05-05 15:54:05,826 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] compactions.Compactor(207): Compacting 9c1ddf63621a4929801ae3ae5ecd96f6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683302039175 2023-05-05 15:54:05,826 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] compactions.Compactor(207): Compacting bff65cd94e8144b7ad8e4973e9786c62, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683302039175 2023-05-05 15:54:05,827 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] compactions.Compactor(207): Compacting a9049db5dc314b64acc5536e55dc3800, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683302039366 2023-05-05 15:54:05,827 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] compactions.Compactor(207): Compacting 5a5ecf258e234499923e5da1b21d9ae5, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683302039366 2023-05-05 15:54:05,827 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] compactions.Compactor(207): Compacting 4441f0039a044a6b9af4232fd7b84326, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683302039561 2023-05-05 15:54:05,828 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] compactions.Compactor(207): Compacting 7aacee3dad2e4f21a38483dab3b0197d, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683302039561 2023-05-05 15:54:05,828 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-05-05 15:54:05,828 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 in 235 msec 2023-05-05 15:54:05,830 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN in 1.5630 sec 2023-05-05 15:54:05,841 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-05-05 15:54:05,849 DEBUG [Listener at localhost/35117] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-05-05 15:54:05,851 INFO [Listener at localhost/35117] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-05-05 15:54:05,858 INFO [Listener at localhost/35117] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-05-05 15:54:05,858 INFO [Listener at localhost/35117] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-05-05 15:54:05,859 INFO [Listener at localhost/35117] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-05-05 15:54:05,866 INFO [RS:1;jenkins-hbase4:42799-shortCompactions-0] throttle.PressureAwareThroughputController(145): b299eb879fd7ed0f63e4b91ba5268da3#cf1#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:54:05,873 INFO [RS:1;jenkins-hbase4:42799-longCompactions-0] throttle.PressureAwareThroughputController(145): b299eb879fd7ed0f63e4b91ba5268da3#cf2#compaction#51 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:54:05,898 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf1/42168f814f654daa875298455935c325 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/42168f814f654daa875298455935c325 2023-05-05 15:54:05,902 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/.tmp/cf2/15dd5a53e04d41f5aeb0ffd8d2550e8b as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/15dd5a53e04d41f5aeb0ffd8d2550e8b 2023-05-05 15:54:05,909 INFO [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b299eb879fd7ed0f63e4b91ba5268da3/cf1 of b299eb879fd7ed0f63e4b91ba5268da3 into 42168f814f654daa875298455935c325(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:54:05,909 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for b299eb879fd7ed0f63e4b91ba5268da3: 2023-05-05 15:54:05,909 INFO [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3., storeName=b299eb879fd7ed0f63e4b91ba5268da3/cf1, priority=9, startTime=1683302045821; duration=0sec 2023-05-05 15:54:05,910 DEBUG [RS:1;jenkins-hbase4:42799-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:54:05,914 INFO [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b299eb879fd7ed0f63e4b91ba5268da3/cf2 of b299eb879fd7ed0f63e4b91ba5268da3 into 15dd5a53e04d41f5aeb0ffd8d2550e8b(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:54:05,915 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for b299eb879fd7ed0f63e4b91ba5268da3: 2023-05-05 15:54:05,915 INFO [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3., storeName=b299eb879fd7ed0f63e4b91ba5268da3/cf2, priority=9, startTime=1683302045822; duration=0sec 2023-05-05 15:54:05,915 DEBUG [RS:1;jenkins-hbase4:42799-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:54:05,923 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-05 15:54:05,925 INFO [Listener at localhost/35117] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-05-05 15:54:05,925 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRollbackAndDoubleExecution 2023-05-05 15:54:05,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:05,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-05-05 15:54:05,929 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302045929"}]},"ts":"1683302045929"} 2023-05-05 15:54:05,931 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-05-05 15:54:05,932 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-05-05 15:54:05,933 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN}] 2023-05-05 15:54:05,934 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN 2023-05-05 15:54:05,935 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=b299eb879fd7ed0f63e4b91ba5268da3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:05,935 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683302045934"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302045934"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302045934"}]},"ts":"1683302045934"} 2023-05-05 15:54:05,936 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977}] 2023-05-05 15:54:06,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-05-05 15:54:06,088 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:06,088 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b299eb879fd7ed0f63e4b91ba5268da3, disabling compactions & flushes 2023-05-05 15:54:06,088 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:54:06,088 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:54:06,088 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. after waiting 0 ms 2023-05-05 15:54:06,088 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:54:06,089 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/9eac5d62b79f49159581b82e191d6f4a, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/a79dfb5adfe544bd973d9313c282fdcb, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/9ce680e0fa604098a7aa8ac4b9b7166c, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/8639cc4946f44be3a5974d15ad612054, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/bff65cd94e8144b7ad8e4973e9786c62, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/5a5ecf258e234499923e5da1b21d9ae5, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/7aacee3dad2e4f21a38483dab3b0197d] to archive 2023-05-05 15:54:06,090 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-05 15:54:06,092 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/9eac5d62b79f49159581b82e191d6f4a to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/9eac5d62b79f49159581b82e191d6f4a 2023-05-05 15:54:06,093 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/a79dfb5adfe544bd973d9313c282fdcb to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/a79dfb5adfe544bd973d9313c282fdcb 2023-05-05 15:54:06,094 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/9ce680e0fa604098a7aa8ac4b9b7166c to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/9ce680e0fa604098a7aa8ac4b9b7166c 2023-05-05 15:54:06,095 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/8639cc4946f44be3a5974d15ad612054 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/8639cc4946f44be3a5974d15ad612054 2023-05-05 15:54:06,097 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/bff65cd94e8144b7ad8e4973e9786c62 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/bff65cd94e8144b7ad8e4973e9786c62 2023-05-05 15:54:06,098 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/5a5ecf258e234499923e5da1b21d9ae5 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/5a5ecf258e234499923e5da1b21d9ae5 2023-05-05 15:54:06,100 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/7aacee3dad2e4f21a38483dab3b0197d to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/7aacee3dad2e4f21a38483dab3b0197d 2023-05-05 15:54:06,104 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/1ea30cc7aa0f49b4bb29289547c266bb, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/8502d9b6ae874806a9213be374c9565d, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/fd1f211bab184c0a9df23ab9a2232ea2, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/4d2917f1d2514972a7d495e162a2584d, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/9c1ddf63621a4929801ae3ae5ecd96f6, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/a9049db5dc314b64acc5536e55dc3800, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/4441f0039a044a6b9af4232fd7b84326] to archive 2023-05-05 15:54:06,104 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-05 15:54:06,106 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/1ea30cc7aa0f49b4bb29289547c266bb to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/1ea30cc7aa0f49b4bb29289547c266bb 2023-05-05 15:54:06,107 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/8502d9b6ae874806a9213be374c9565d to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/8502d9b6ae874806a9213be374c9565d 2023-05-05 15:54:06,108 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/fd1f211bab184c0a9df23ab9a2232ea2 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/fd1f211bab184c0a9df23ab9a2232ea2 2023-05-05 15:54:06,109 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/4d2917f1d2514972a7d495e162a2584d to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/4d2917f1d2514972a7d495e162a2584d 2023-05-05 15:54:06,110 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/9c1ddf63621a4929801ae3ae5ecd96f6 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/9c1ddf63621a4929801ae3ae5ecd96f6 2023-05-05 15:54:06,111 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/a9049db5dc314b64acc5536e55dc3800 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/a9049db5dc314b64acc5536e55dc3800 2023-05-05 15:54:06,113 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/4441f0039a044a6b9af4232fd7b84326 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/4441f0039a044a6b9af4232fd7b84326 2023-05-05 15:54:06,116 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-05 15:54:06,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:54:06,117 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3. 2023-05-05 15:54:06,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b299eb879fd7ed0f63e4b91ba5268da3: 2023-05-05 15:54:06,118 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:06,119 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=b299eb879fd7ed0f63e4b91ba5268da3, regionState=CLOSED 2023-05-05 15:54:06,119 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683302046119"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302046119"}]},"ts":"1683302046119"} 2023-05-05 15:54:06,123 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-05-05 15:54:06,123 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure b299eb879fd7ed0f63e4b91ba5268da3, server=jenkins-hbase4.apache.org,42799,1683302022977 in 185 msec 2023-05-05 15:54:06,125 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-05-05 15:54:06,125 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, UNASSIGN in 190 msec 2023-05-05 15:54:06,125 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302046125"}]},"ts":"1683302046125"} 2023-05-05 15:54:06,127 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-05-05 15:54:06,128 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-05-05 15:54:06,130 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 204 msec 2023-05-05 15:54:06,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-05-05 15:54:06,232 INFO [Listener at localhost/35117] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-05-05 15:54:06,232 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRollbackAndDoubleExecution 2023-05-05 15:54:06,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:06,235 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:06,236 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:06,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-05-05 15:54:06,239 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:06,241 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/recovered.edits] 2023-05-05 15:54:06,244 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/42168f814f654daa875298455935c325 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf1/42168f814f654daa875298455935c325 2023-05-05 15:54:06,246 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/15dd5a53e04d41f5aeb0ffd8d2550e8b to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/cf2/15dd5a53e04d41f5aeb0ffd8d2550e8b 2023-05-05 15:54:06,249 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/recovered.edits/90.seqid to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3/recovered.edits/90.seqid 2023-05-05 15:54:06,250 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRollbackAndDoubleExecution/b299eb879fd7ed0f63e4b91ba5268da3 2023-05-05 15:54:06,250 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-05-05 15:54:06,253 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:06,255 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-05-05 15:54:06,256 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-05-05 15:54:06,257 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:06,257 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-05-05 15:54:06,257 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302046257"}]},"ts":"9223372036854775807"} 2023-05-05 15:54:06,259 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-05-05 15:54:06,259 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => b299eb879fd7ed0f63e4b91ba5268da3, NAME => 'testRollbackAndDoubleExecution,,1683302037876.b299eb879fd7ed0f63e4b91ba5268da3.', STARTKEY => '', ENDKEY => ''}] 2023-05-05 15:54:06,259 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-05-05 15:54:06,259 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683302046259"}]},"ts":"9223372036854775807"} 2023-05-05 15:54:06,260 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-05-05 15:54:06,262 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:54:06,263 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 29 msec 2023-05-05 15:54:06,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-05-05 15:54:06,337 INFO [Listener at localhost/35117] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-05-05 15:54:06,354 INFO [Listener at localhost/35117] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=399 (was 402), OpenFileDescriptor=841 (was 868), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=307 (was 335), ProcessCount=179 (was 179), AvailableMemoryMB=5217 (was 5315) 2023-05-05 15:54:06,370 INFO [Listener at localhost/35117] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=399, OpenFileDescriptor=841, MaxFileDescriptor=60000, SystemLoadAverage=307, ProcessCount=179, AvailableMemoryMB=5216 2023-05-05 15:54:06,370 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-05 15:54:06,374 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-05 15:54:06,374 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-05 15:54:06,375 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:54:06,377 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-05-05 15:54:06,378 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-05 15:54:06,379 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionNoStoreFile/cfa9eb043f55be3f9ab828cd40416e7f 2023-05-05 15:54:06,380 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionNoStoreFile/cfa9eb043f55be3f9ab828cd40416e7f empty. 2023-05-05 15:54:06,380 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionNoStoreFile/cfa9eb043f55be3f9ab828cd40416e7f 2023-05-05 15:54:06,380 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-05-05 15:54:06,391 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-05-05 15:54:06,392 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7675): creating {ENCODED => cfa9eb043f55be3f9ab828cd40416e7f, NAME => 'testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp 2023-05-05 15:54:06,407 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:54:06,407 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1604): Closing cfa9eb043f55be3f9ab828cd40416e7f, disabling compactions & flushes 2023-05-05 15:54:06,407 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f. 2023-05-05 15:54:06,407 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f. 2023-05-05 15:54:06,407 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f. after waiting 0 ms 2023-05-05 15:54:06,407 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f. 2023-05-05 15:54:06,407 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f. 2023-05-05 15:54:06,407 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1558): Region close journal for cfa9eb043f55be3f9ab828cd40416e7f: 2023-05-05 15:54:06,409 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-05-05 15:54:06,410 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1683302046410"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302046410"}]},"ts":"1683302046410"} 2023-05-05 15:54:06,411 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-05 15:54:06,412 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-05 15:54:06,412 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302046412"}]},"ts":"1683302046412"} 2023-05-05 15:54:06,414 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-05-05 15:54:06,417 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-05 15:54:06,417 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-05 15:54:06,417 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-05 15:54:06,417 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-05 15:54:06,417 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-05 15:54:06,418 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=cfa9eb043f55be3f9ab828cd40416e7f, ASSIGN}] 2023-05-05 15:54:06,419 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=cfa9eb043f55be3f9ab828cd40416e7f, ASSIGN 2023-05-05 15:54:06,420 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=cfa9eb043f55be3f9ab828cd40416e7f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42799,1683302022977; forceNewPlan=false, retain=false 2023-05-05 15:54:06,570 INFO [jenkins-hbase4:44213] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-05 15:54:06,571 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=cfa9eb043f55be3f9ab828cd40416e7f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:06,571 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1683302046571"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302046571"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302046571"}]},"ts":"1683302046571"} 2023-05-05 15:54:06,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure cfa9eb043f55be3f9ab828cd40416e7f, server=jenkins-hbase4.apache.org,42799,1683302022977}] 2023-05-05 15:54:06,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f. 2023-05-05 15:54:06,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => cfa9eb043f55be3f9ab828cd40416e7f, NAME => 'testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f.', STARTKEY => '', ENDKEY => ''} 2023-05-05 15:54:06,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:54:06,729 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile cfa9eb043f55be3f9ab828cd40416e7f 2023-05-05 15:54:06,729 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:54:06,729 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for cfa9eb043f55be3f9ab828cd40416e7f 2023-05-05 15:54:06,729 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for cfa9eb043f55be3f9ab828cd40416e7f 2023-05-05 15:54:06,730 INFO [StoreOpener-cfa9eb043f55be3f9ab828cd40416e7f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region cfa9eb043f55be3f9ab828cd40416e7f 2023-05-05 15:54:06,731 DEBUG [StoreOpener-cfa9eb043f55be3f9ab828cd40416e7f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionNoStoreFile/cfa9eb043f55be3f9ab828cd40416e7f/cf1 2023-05-05 15:54:06,732 DEBUG [StoreOpener-cfa9eb043f55be3f9ab828cd40416e7f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionNoStoreFile/cfa9eb043f55be3f9ab828cd40416e7f/cf1 2023-05-05 15:54:06,732 INFO [StoreOpener-cfa9eb043f55be3f9ab828cd40416e7f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cfa9eb043f55be3f9ab828cd40416e7f columnFamilyName cf1 2023-05-05 15:54:06,733 INFO [StoreOpener-cfa9eb043f55be3f9ab828cd40416e7f-1] regionserver.HStore(310): Store=cfa9eb043f55be3f9ab828cd40416e7f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:54:06,733 INFO [StoreOpener-cfa9eb043f55be3f9ab828cd40416e7f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region cfa9eb043f55be3f9ab828cd40416e7f 2023-05-05 15:54:06,735 DEBUG [StoreOpener-cfa9eb043f55be3f9ab828cd40416e7f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionNoStoreFile/cfa9eb043f55be3f9ab828cd40416e7f/cf2 2023-05-05 15:54:06,735 DEBUG [StoreOpener-cfa9eb043f55be3f9ab828cd40416e7f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionNoStoreFile/cfa9eb043f55be3f9ab828cd40416e7f/cf2 2023-05-05 15:54:06,735 INFO [StoreOpener-cfa9eb043f55be3f9ab828cd40416e7f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cfa9eb043f55be3f9ab828cd40416e7f columnFamilyName cf2 2023-05-05 15:54:06,736 INFO [StoreOpener-cfa9eb043f55be3f9ab828cd40416e7f-1] regionserver.HStore(310): Store=cfa9eb043f55be3f9ab828cd40416e7f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:54:06,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionNoStoreFile/cfa9eb043f55be3f9ab828cd40416e7f 2023-05-05 15:54:06,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionNoStoreFile/cfa9eb043f55be3f9ab828cd40416e7f 2023-05-05 15:54:06,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-05 15:54:06,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for cfa9eb043f55be3f9ab828cd40416e7f 2023-05-05 15:54:06,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionNoStoreFile/cfa9eb043f55be3f9ab828cd40416e7f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-05 15:54:06,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened cfa9eb043f55be3f9ab828cd40416e7f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69099655, jitterRate=0.02966509759426117}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-05 15:54:06,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for cfa9eb043f55be3f9ab828cd40416e7f: 2023-05-05 15:54:06,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f., pid=55, masterSystemTime=1683302046725 2023-05-05 15:54:06,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f. 2023-05-05 15:54:06,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f. 2023-05-05 15:54:06,747 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=cfa9eb043f55be3f9ab828cd40416e7f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:06,747 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1683302046747"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302046747"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302046747"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302046747"}]},"ts":"1683302046747"} 2023-05-05 15:54:06,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-05-05 15:54:06,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure cfa9eb043f55be3f9ab828cd40416e7f, server=jenkins-hbase4.apache.org,42799,1683302022977 in 176 msec 2023-05-05 15:54:06,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-05-05 15:54:06,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=cfa9eb043f55be3f9ab828cd40416e7f, ASSIGN in 333 msec 2023-05-05 15:54:06,753 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-05 15:54:06,753 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302046753"}]},"ts":"1683302046753"} 2023-05-05 15:54:06,754 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-05-05 15:54:06,756 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-05-05 15:54:06,758 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 381 msec 2023-05-05 15:54:06,877 DEBUG [Listener at localhost/35117] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:06,878 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=cfa9eb043f55be3f9ab828cd40416e7f, daughterA=3296930d4aa28614ec2511a0a3f26d3c, daughterB=c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:54:06,879 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=cfa9eb043f55be3f9ab828cd40416e7f, daughterA=3296930d4aa28614ec2511a0a3f26d3c, daughterB=c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:54:06,879 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=cfa9eb043f55be3f9ab828cd40416e7f, daughterA=3296930d4aa28614ec2511a0a3f26d3c, daughterB=c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:54:06,879 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=cfa9eb043f55be3f9ab828cd40416e7f, daughterA=3296930d4aa28614ec2511a0a3f26d3c, daughterB=c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:54:06,881 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=cfa9eb043f55be3f9ab828cd40416e7f, UNASSIGN}] 2023-05-05 15:54:06,881 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=cfa9eb043f55be3f9ab828cd40416e7f, UNASSIGN 2023-05-05 15:54:06,882 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=cfa9eb043f55be3f9ab828cd40416e7f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:06,882 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1683302046882"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302046882"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302046882"}]},"ts":"1683302046882"} 2023-05-05 15:54:06,883 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure cfa9eb043f55be3f9ab828cd40416e7f, server=jenkins-hbase4.apache.org,42799,1683302022977}] 2023-05-05 15:54:07,035 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close cfa9eb043f55be3f9ab828cd40416e7f 2023-05-05 15:54:07,036 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing cfa9eb043f55be3f9ab828cd40416e7f, disabling compactions & flushes 2023-05-05 15:54:07,036 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f. 2023-05-05 15:54:07,037 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f. 2023-05-05 15:54:07,037 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f. after waiting 0 ms 2023-05-05 15:54:07,037 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f. 2023-05-05 15:54:07,040 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionNoStoreFile/cfa9eb043f55be3f9ab828cd40416e7f/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-05-05 15:54:07,041 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:54:07,041 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f. 2023-05-05 15:54:07,041 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for cfa9eb043f55be3f9ab828cd40416e7f: 2023-05-05 15:54:07,043 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed cfa9eb043f55be3f9ab828cd40416e7f 2023-05-05 15:54:07,044 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=cfa9eb043f55be3f9ab828cd40416e7f, regionState=CLOSED 2023-05-05 15:54:07,044 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1683302047043"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302047043"}]},"ts":"1683302047043"} 2023-05-05 15:54:07,047 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-05-05 15:54:07,047 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure cfa9eb043f55be3f9ab828cd40416e7f, server=jenkins-hbase4.apache.org,42799,1683302022977 in 162 msec 2023-05-05 15:54:07,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-05-05 15:54:07,049 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=cfa9eb043f55be3f9ab828cd40416e7f, UNASSIGN in 166 msec 2023-05-05 15:54:07,092 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionNoStoreFile/3296930d4aa28614ec2511a0a3f26d3c/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-05-05 15:54:07,094 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionNoStoreFile/c4477a15aabbb6987e68dcac21577a7f/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-05-05 15:54:07,096 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1683302047096"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1683302047096"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1683302047096"}]},"ts":"1683302047096"} 2023-05-05 15:54:07,096 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1683302046877.3296930d4aa28614ec2511a0a3f26d3c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683302047096"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302047096"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302047096"}]},"ts":"1683302047096"} 2023-05-05 15:54:07,096 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1683302046877.c4477a15aabbb6987e68dcac21577a7f.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683302047096"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302047096"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302047096"}]},"ts":"1683302047096"} 2023-05-05 15:54:07,099 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3296930d4aa28614ec2511a0a3f26d3c, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=c4477a15aabbb6987e68dcac21577a7f, ASSIGN}] 2023-05-05 15:54:07,100 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=c4477a15aabbb6987e68dcac21577a7f, ASSIGN 2023-05-05 15:54:07,101 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=c4477a15aabbb6987e68dcac21577a7f, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42799,1683302022977; forceNewPlan=false, retain=false 2023-05-05 15:54:07,102 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3296930d4aa28614ec2511a0a3f26d3c, ASSIGN 2023-05-05 15:54:07,102 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3296930d4aa28614ec2511a0a3f26d3c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42799,1683302022977; forceNewPlan=false, retain=false 2023-05-05 15:54:07,251 INFO [jenkins-hbase4:44213] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-05 15:54:07,254 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=c4477a15aabbb6987e68dcac21577a7f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:07,254 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1683302046877.c4477a15aabbb6987e68dcac21577a7f.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683302047253"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302047253"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302047253"}]},"ts":"1683302047253"} 2023-05-05 15:54:07,256 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE; OpenRegionProcedure c4477a15aabbb6987e68dcac21577a7f, server=jenkins-hbase4.apache.org,42799,1683302022977}] 2023-05-05 15:54:07,259 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=3296930d4aa28614ec2511a0a3f26d3c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:07,259 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1683302046877.3296930d4aa28614ec2511a0a3f26d3c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683302047259"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302047259"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302047259"}]},"ts":"1683302047259"} 2023-05-05 15:54:07,260 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=59, state=RUNNABLE; OpenRegionProcedure 3296930d4aa28614ec2511a0a3f26d3c, server=jenkins-hbase4.apache.org,42799,1683302022977}] 2023-05-05 15:54:07,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,,1683302046877.3296930d4aa28614ec2511a0a3f26d3c. 2023-05-05 15:54:07,411 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3296930d4aa28614ec2511a0a3f26d3c, NAME => 'testSplitTableRegionNoStoreFile,,1683302046877.3296930d4aa28614ec2511a0a3f26d3c.', STARTKEY => '', ENDKEY => '41'} 2023-05-05 15:54:07,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:54:07,411 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 3296930d4aa28614ec2511a0a3f26d3c 2023-05-05 15:54:07,411 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1683302046877.3296930d4aa28614ec2511a0a3f26d3c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:54:07,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3296930d4aa28614ec2511a0a3f26d3c 2023-05-05 15:54:07,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3296930d4aa28614ec2511a0a3f26d3c 2023-05-05 15:54:07,413 INFO [StoreOpener-3296930d4aa28614ec2511a0a3f26d3c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3296930d4aa28614ec2511a0a3f26d3c 2023-05-05 15:54:07,414 DEBUG [StoreOpener-3296930d4aa28614ec2511a0a3f26d3c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionNoStoreFile/3296930d4aa28614ec2511a0a3f26d3c/cf1 2023-05-05 15:54:07,414 DEBUG [StoreOpener-3296930d4aa28614ec2511a0a3f26d3c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionNoStoreFile/3296930d4aa28614ec2511a0a3f26d3c/cf1 2023-05-05 15:54:07,415 INFO [StoreOpener-3296930d4aa28614ec2511a0a3f26d3c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3296930d4aa28614ec2511a0a3f26d3c columnFamilyName cf1 2023-05-05 15:54:07,415 INFO [StoreOpener-3296930d4aa28614ec2511a0a3f26d3c-1] regionserver.HStore(310): Store=3296930d4aa28614ec2511a0a3f26d3c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:54:07,415 INFO [StoreOpener-3296930d4aa28614ec2511a0a3f26d3c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3296930d4aa28614ec2511a0a3f26d3c 2023-05-05 15:54:07,417 DEBUG [StoreOpener-3296930d4aa28614ec2511a0a3f26d3c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionNoStoreFile/3296930d4aa28614ec2511a0a3f26d3c/cf2 2023-05-05 15:54:07,417 DEBUG [StoreOpener-3296930d4aa28614ec2511a0a3f26d3c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionNoStoreFile/3296930d4aa28614ec2511a0a3f26d3c/cf2 2023-05-05 15:54:07,417 INFO [StoreOpener-3296930d4aa28614ec2511a0a3f26d3c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3296930d4aa28614ec2511a0a3f26d3c columnFamilyName cf2 2023-05-05 15:54:07,418 INFO [StoreOpener-3296930d4aa28614ec2511a0a3f26d3c-1] regionserver.HStore(310): Store=3296930d4aa28614ec2511a0a3f26d3c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:54:07,418 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionNoStoreFile/3296930d4aa28614ec2511a0a3f26d3c 2023-05-05 15:54:07,420 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionNoStoreFile/3296930d4aa28614ec2511a0a3f26d3c 2023-05-05 15:54:07,421 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-05 15:54:07,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3296930d4aa28614ec2511a0a3f26d3c 2023-05-05 15:54:07,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3296930d4aa28614ec2511a0a3f26d3c; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73226223, jitterRate=0.0911557525396347}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-05 15:54:07,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3296930d4aa28614ec2511a0a3f26d3c: 2023-05-05 15:54:07,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1683302046877.3296930d4aa28614ec2511a0a3f26d3c., pid=62, masterSystemTime=1683302047408 2023-05-05 15:54:07,426 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1683302046877.3296930d4aa28614ec2511a0a3f26d3c. 2023-05-05 15:54:07,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,,1683302046877.3296930d4aa28614ec2511a0a3f26d3c. 2023-05-05 15:54:07,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,41,1683302046877.c4477a15aabbb6987e68dcac21577a7f. 2023-05-05 15:54:07,426 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c4477a15aabbb6987e68dcac21577a7f, NAME => 'testSplitTableRegionNoStoreFile,41,1683302046877.c4477a15aabbb6987e68dcac21577a7f.', STARTKEY => '41', ENDKEY => ''} 2023-05-05 15:54:07,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:54:07,426 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=3296930d4aa28614ec2511a0a3f26d3c, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:07,426 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:54:07,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,41,1683302046877.c4477a15aabbb6987e68dcac21577a7f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:54:07,427 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1683302046877.3296930d4aa28614ec2511a0a3f26d3c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683302047426"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302047426"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302047426"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302047426"}]},"ts":"1683302047426"} 2023-05-05 15:54:07,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:54:07,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:54:07,428 INFO [StoreOpener-c4477a15aabbb6987e68dcac21577a7f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:54:07,430 DEBUG [StoreOpener-c4477a15aabbb6987e68dcac21577a7f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionNoStoreFile/c4477a15aabbb6987e68dcac21577a7f/cf1 2023-05-05 15:54:07,430 DEBUG [StoreOpener-c4477a15aabbb6987e68dcac21577a7f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionNoStoreFile/c4477a15aabbb6987e68dcac21577a7f/cf1 2023-05-05 15:54:07,430 INFO [StoreOpener-c4477a15aabbb6987e68dcac21577a7f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c4477a15aabbb6987e68dcac21577a7f columnFamilyName cf1 2023-05-05 15:54:07,431 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=59 2023-05-05 15:54:07,431 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=59, state=SUCCESS; OpenRegionProcedure 3296930d4aa28614ec2511a0a3f26d3c, server=jenkins-hbase4.apache.org,42799,1683302022977 in 168 msec 2023-05-05 15:54:07,431 INFO [StoreOpener-c4477a15aabbb6987e68dcac21577a7f-1] regionserver.HStore(310): Store=c4477a15aabbb6987e68dcac21577a7f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:54:07,431 INFO [StoreOpener-c4477a15aabbb6987e68dcac21577a7f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:54:07,432 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3296930d4aa28614ec2511a0a3f26d3c, ASSIGN in 332 msec 2023-05-05 15:54:07,433 DEBUG [StoreOpener-c4477a15aabbb6987e68dcac21577a7f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionNoStoreFile/c4477a15aabbb6987e68dcac21577a7f/cf2 2023-05-05 15:54:07,433 DEBUG [StoreOpener-c4477a15aabbb6987e68dcac21577a7f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionNoStoreFile/c4477a15aabbb6987e68dcac21577a7f/cf2 2023-05-05 15:54:07,433 INFO [StoreOpener-c4477a15aabbb6987e68dcac21577a7f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c4477a15aabbb6987e68dcac21577a7f columnFamilyName cf2 2023-05-05 15:54:07,434 INFO [StoreOpener-c4477a15aabbb6987e68dcac21577a7f-1] regionserver.HStore(310): Store=c4477a15aabbb6987e68dcac21577a7f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:54:07,434 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionNoStoreFile/c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:54:07,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionNoStoreFile/c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:54:07,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-05 15:54:07,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:54:07,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c4477a15aabbb6987e68dcac21577a7f; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72948015, jitterRate=0.08701013028621674}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-05 15:54:07,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c4477a15aabbb6987e68dcac21577a7f: 2023-05-05 15:54:07,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1683302046877.c4477a15aabbb6987e68dcac21577a7f., pid=61, masterSystemTime=1683302047408 2023-05-05 15:54:07,442 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1683302046877.c4477a15aabbb6987e68dcac21577a7f. 2023-05-05 15:54:07,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,41,1683302046877.c4477a15aabbb6987e68dcac21577a7f. 2023-05-05 15:54:07,443 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=c4477a15aabbb6987e68dcac21577a7f, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:07,443 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1683302046877.c4477a15aabbb6987e68dcac21577a7f.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683302047443"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302047443"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302047443"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302047443"}]},"ts":"1683302047443"} 2023-05-05 15:54:07,446 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-05-05 15:54:07,446 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; OpenRegionProcedure c4477a15aabbb6987e68dcac21577a7f, server=jenkins-hbase4.apache.org,42799,1683302022977 in 188 msec 2023-05-05 15:54:07,448 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=56 2023-05-05 15:54:07,448 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=c4477a15aabbb6987e68dcac21577a7f, ASSIGN in 347 msec 2023-05-05 15:54:07,449 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=cfa9eb043f55be3f9ab828cd40416e7f, daughterA=3296930d4aa28614ec2511a0a3f26d3c, daughterB=c4477a15aabbb6987e68dcac21577a7f in 571 msec 2023-05-05 15:54:07,633 DEBUG [Listener at localhost/35117] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-05 15:54:07,636 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-05 15:54:07,638 INFO [Listener at localhost/35117] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-05-05 15:54:07,638 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionNoStoreFile 2023-05-05 15:54:07,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:54:07,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-05-05 15:54:07,642 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302047642"}]},"ts":"1683302047642"} 2023-05-05 15:54:07,643 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-05-05 15:54:07,646 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-05-05 15:54:07,647 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3296930d4aa28614ec2511a0a3f26d3c, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=c4477a15aabbb6987e68dcac21577a7f, UNASSIGN}] 2023-05-05 15:54:07,648 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=c4477a15aabbb6987e68dcac21577a7f, UNASSIGN 2023-05-05 15:54:07,648 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=c4477a15aabbb6987e68dcac21577a7f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:07,648 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1683302046877.c4477a15aabbb6987e68dcac21577a7f.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683302047648"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302047648"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302047648"}]},"ts":"1683302047648"} 2023-05-05 15:54:07,649 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure c4477a15aabbb6987e68dcac21577a7f, server=jenkins-hbase4.apache.org,42799,1683302022977}] 2023-05-05 15:54:07,652 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3296930d4aa28614ec2511a0a3f26d3c, UNASSIGN 2023-05-05 15:54:07,652 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=3296930d4aa28614ec2511a0a3f26d3c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:07,652 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1683302046877.3296930d4aa28614ec2511a0a3f26d3c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683302047652"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302047652"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302047652"}]},"ts":"1683302047652"} 2023-05-05 15:54:07,653 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 3296930d4aa28614ec2511a0a3f26d3c, server=jenkins-hbase4.apache.org,42799,1683302022977}] 2023-05-05 15:54:07,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-05-05 15:54:07,801 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:54:07,802 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c4477a15aabbb6987e68dcac21577a7f, disabling compactions & flushes 2023-05-05 15:54:07,802 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,41,1683302046877.c4477a15aabbb6987e68dcac21577a7f. 2023-05-05 15:54:07,802 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1683302046877.c4477a15aabbb6987e68dcac21577a7f. 2023-05-05 15:54:07,803 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,41,1683302046877.c4477a15aabbb6987e68dcac21577a7f. after waiting 0 ms 2023-05-05 15:54:07,803 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,41,1683302046877.c4477a15aabbb6987e68dcac21577a7f. 2023-05-05 15:54:07,806 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionNoStoreFile/c4477a15aabbb6987e68dcac21577a7f/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-05-05 15:54:07,807 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:54:07,807 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,41,1683302046877.c4477a15aabbb6987e68dcac21577a7f. 2023-05-05 15:54:07,807 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c4477a15aabbb6987e68dcac21577a7f: 2023-05-05 15:54:07,808 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:54:07,808 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 3296930d4aa28614ec2511a0a3f26d3c 2023-05-05 15:54:07,809 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3296930d4aa28614ec2511a0a3f26d3c, disabling compactions & flushes 2023-05-05 15:54:07,809 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1683302046877.3296930d4aa28614ec2511a0a3f26d3c. 2023-05-05 15:54:07,809 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1683302046877.3296930d4aa28614ec2511a0a3f26d3c. 2023-05-05 15:54:07,809 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1683302046877.3296930d4aa28614ec2511a0a3f26d3c. after waiting 0 ms 2023-05-05 15:54:07,809 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1683302046877.3296930d4aa28614ec2511a0a3f26d3c. 2023-05-05 15:54:07,809 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=c4477a15aabbb6987e68dcac21577a7f, regionState=CLOSED 2023-05-05 15:54:07,809 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1683302046877.c4477a15aabbb6987e68dcac21577a7f.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683302047809"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302047809"}]},"ts":"1683302047809"} 2023-05-05 15:54:07,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionNoStoreFile/3296930d4aa28614ec2511a0a3f26d3c/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-05-05 15:54:07,812 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-05-05 15:54:07,812 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure c4477a15aabbb6987e68dcac21577a7f, server=jenkins-hbase4.apache.org,42799,1683302022977 in 162 msec 2023-05-05 15:54:07,813 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:54:07,813 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1683302046877.3296930d4aa28614ec2511a0a3f26d3c. 2023-05-05 15:54:07,813 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3296930d4aa28614ec2511a0a3f26d3c: 2023-05-05 15:54:07,814 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=c4477a15aabbb6987e68dcac21577a7f, UNASSIGN in 166 msec 2023-05-05 15:54:07,814 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 3296930d4aa28614ec2511a0a3f26d3c 2023-05-05 15:54:07,815 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=3296930d4aa28614ec2511a0a3f26d3c, regionState=CLOSED 2023-05-05 15:54:07,815 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1683302046877.3296930d4aa28614ec2511a0a3f26d3c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683302047815"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302047815"}]},"ts":"1683302047815"} 2023-05-05 15:54:07,817 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-05-05 15:54:07,817 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 3296930d4aa28614ec2511a0a3f26d3c, server=jenkins-hbase4.apache.org,42799,1683302022977 in 163 msec 2023-05-05 15:54:07,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=64, resume processing ppid=63 2023-05-05 15:54:07,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3296930d4aa28614ec2511a0a3f26d3c, UNASSIGN in 170 msec 2023-05-05 15:54:07,819 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302047819"}]},"ts":"1683302047819"} 2023-05-05 15:54:07,821 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-05-05 15:54:07,822 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-05-05 15:54:07,825 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 185 msec 2023-05-05 15:54:07,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-05-05 15:54:07,944 INFO [Listener at localhost/35117] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-05-05 15:54:07,945 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionNoStoreFile 2023-05-05 15:54:07,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:54:07,947 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:54:07,948 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:54:07,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-05-05 15:54:07,954 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionNoStoreFile/cfa9eb043f55be3f9ab828cd40416e7f 2023-05-05 15:54:07,954 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionNoStoreFile/c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:54:07,954 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionNoStoreFile/3296930d4aa28614ec2511a0a3f26d3c 2023-05-05 15:54:07,956 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionNoStoreFile/c4477a15aabbb6987e68dcac21577a7f/cf1, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionNoStoreFile/c4477a15aabbb6987e68dcac21577a7f/cf2, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionNoStoreFile/c4477a15aabbb6987e68dcac21577a7f/recovered.edits] 2023-05-05 15:54:07,957 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionNoStoreFile/3296930d4aa28614ec2511a0a3f26d3c/cf1, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionNoStoreFile/3296930d4aa28614ec2511a0a3f26d3c/cf2, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionNoStoreFile/3296930d4aa28614ec2511a0a3f26d3c/recovered.edits] 2023-05-05 15:54:07,957 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionNoStoreFile/cfa9eb043f55be3f9ab828cd40416e7f/cf1, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionNoStoreFile/cfa9eb043f55be3f9ab828cd40416e7f/cf2, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionNoStoreFile/cfa9eb043f55be3f9ab828cd40416e7f/recovered.edits] 2023-05-05 15:54:07,964 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionNoStoreFile/c4477a15aabbb6987e68dcac21577a7f/recovered.edits/7.seqid to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionNoStoreFile/c4477a15aabbb6987e68dcac21577a7f/recovered.edits/7.seqid 2023-05-05 15:54:07,965 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionNoStoreFile/3296930d4aa28614ec2511a0a3f26d3c/recovered.edits/7.seqid to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionNoStoreFile/3296930d4aa28614ec2511a0a3f26d3c/recovered.edits/7.seqid 2023-05-05 15:54:07,966 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionNoStoreFile/c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:54:07,966 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionNoStoreFile/cfa9eb043f55be3f9ab828cd40416e7f/recovered.edits/4.seqid to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionNoStoreFile/cfa9eb043f55be3f9ab828cd40416e7f/recovered.edits/4.seqid 2023-05-05 15:54:07,966 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionNoStoreFile/3296930d4aa28614ec2511a0a3f26d3c 2023-05-05 15:54:07,967 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionNoStoreFile/cfa9eb043f55be3f9ab828cd40416e7f 2023-05-05 15:54:07,967 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-05-05 15:54:07,969 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:54:07,972 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-05-05 15:54:07,974 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-05-05 15:54:07,975 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:54:07,975 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-05-05 15:54:07,975 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302047975"}]},"ts":"9223372036854775807"} 2023-05-05 15:54:07,975 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1683302046877.3296930d4aa28614ec2511a0a3f26d3c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302047975"}]},"ts":"9223372036854775807"} 2023-05-05 15:54:07,975 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1683302046877.c4477a15aabbb6987e68dcac21577a7f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302047975"}]},"ts":"9223372036854775807"} 2023-05-05 15:54:07,976 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-05 15:54:07,977 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => cfa9eb043f55be3f9ab828cd40416e7f, NAME => 'testSplitTableRegionNoStoreFile,,1683302046375.cfa9eb043f55be3f9ab828cd40416e7f.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 3296930d4aa28614ec2511a0a3f26d3c, NAME => 'testSplitTableRegionNoStoreFile,,1683302046877.3296930d4aa28614ec2511a0a3f26d3c.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => c4477a15aabbb6987e68dcac21577a7f, NAME => 'testSplitTableRegionNoStoreFile,41,1683302046877.c4477a15aabbb6987e68dcac21577a7f.', STARTKEY => '41', ENDKEY => ''}] 2023-05-05 15:54:07,977 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-05-05 15:54:07,977 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683302047977"}]},"ts":"9223372036854775807"} 2023-05-05 15:54:07,978 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-05-05 15:54:07,980 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:54:07,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 35 msec 2023-05-05 15:54:08,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-05-05 15:54:08,050 INFO [Listener at localhost/35117] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-05-05 15:54:08,065 INFO [Listener at localhost/35117] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=402 (was 399) Potentially hanging thread: hconnection-0x6277894c-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6277894c-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-963627285_15 at /127.0.0.1:54532 [Waiting for operation #38] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6277894c-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-963627285_15 at /127.0.0.1:59852 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6277894c-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=839 (was 841), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=307 (was 307), ProcessCount=179 (was 179), AvailableMemoryMB=5215 (was 5216) 2023-05-05 15:54:08,079 INFO [Listener at localhost/35117] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=402, OpenFileDescriptor=839, MaxFileDescriptor=60000, SystemLoadAverage=307, ProcessCount=179, AvailableMemoryMB=5214 2023-05-05 15:54:08,079 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-05 15:54:08,081 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-05 15:54:08,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-05 15:54:08,083 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,084 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-05-05 15:54:08,085 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-05 15:54:08,086 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:08,087 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2 empty. 2023-05-05 15:54:08,087 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:08,087 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-05-05 15:54:08,104 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-05-05 15:54:08,105 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => df08d89907b6ac6a28b72ec117993ac2, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp 2023-05-05 15:54:08,115 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:54:08,115 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1604): Closing df08d89907b6ac6a28b72ec117993ac2, disabling compactions & flushes 2023-05-05 15:54:08,115 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:08,115 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:08,115 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. after waiting 0 ms 2023-05-05 15:54:08,115 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:08,115 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:08,115 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1558): Region close journal for df08d89907b6ac6a28b72ec117993ac2: 2023-05-05 15:54:08,117 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-05-05 15:54:08,118 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1683302048118"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302048118"}]},"ts":"1683302048118"} 2023-05-05 15:54:08,119 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-05 15:54:08,120 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-05 15:54:08,120 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302048120"}]},"ts":"1683302048120"} 2023-05-05 15:54:08,121 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-05-05 15:54:08,125 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-05 15:54:08,126 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-05 15:54:08,126 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-05 15:54:08,126 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-05 15:54:08,126 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-05 15:54:08,126 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=df08d89907b6ac6a28b72ec117993ac2, ASSIGN}] 2023-05-05 15:54:08,128 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=df08d89907b6ac6a28b72ec117993ac2, ASSIGN 2023-05-05 15:54:08,128 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=df08d89907b6ac6a28b72ec117993ac2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40991,1683302023016; forceNewPlan=false, retain=false 2023-05-05 15:54:08,278 INFO [jenkins-hbase4:44213] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-05 15:54:08,280 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=df08d89907b6ac6a28b72ec117993ac2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,280 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1683302048280"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302048280"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302048280"}]},"ts":"1683302048280"} 2023-05-05 15:54:08,282 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure df08d89907b6ac6a28b72ec117993ac2, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:54:08,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:08,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => df08d89907b6ac6a28b72ec117993ac2, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2.', STARTKEY => '', ENDKEY => ''} 2023-05-05 15:54:08,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:54:08,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:08,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:54:08,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:08,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:08,441 INFO [StoreOpener-df08d89907b6ac6a28b72ec117993ac2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:08,443 DEBUG [StoreOpener-df08d89907b6ac6a28b72ec117993ac2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1 2023-05-05 15:54:08,443 DEBUG [StoreOpener-df08d89907b6ac6a28b72ec117993ac2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1 2023-05-05 15:54:08,443 INFO [StoreOpener-df08d89907b6ac6a28b72ec117993ac2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region df08d89907b6ac6a28b72ec117993ac2 columnFamilyName cf1 2023-05-05 15:54:08,444 INFO [StoreOpener-df08d89907b6ac6a28b72ec117993ac2-1] regionserver.HStore(310): Store=df08d89907b6ac6a28b72ec117993ac2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:54:08,444 INFO [StoreOpener-df08d89907b6ac6a28b72ec117993ac2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:08,445 DEBUG [StoreOpener-df08d89907b6ac6a28b72ec117993ac2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2 2023-05-05 15:54:08,445 DEBUG [StoreOpener-df08d89907b6ac6a28b72ec117993ac2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2 2023-05-05 15:54:08,445 INFO [StoreOpener-df08d89907b6ac6a28b72ec117993ac2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region df08d89907b6ac6a28b72ec117993ac2 columnFamilyName cf2 2023-05-05 15:54:08,446 INFO [StoreOpener-df08d89907b6ac6a28b72ec117993ac2-1] regionserver.HStore(310): Store=df08d89907b6ac6a28b72ec117993ac2/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:54:08,447 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:08,447 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:08,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-05 15:54:08,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:08,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-05 15:54:08,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened df08d89907b6ac6a28b72ec117993ac2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68455760, jitterRate=0.020070314407348633}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-05 15:54:08,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for df08d89907b6ac6a28b72ec117993ac2: 2023-05-05 15:54:08,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2., pid=71, masterSystemTime=1683302048433 2023-05-05 15:54:08,454 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:08,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:08,454 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=df08d89907b6ac6a28b72ec117993ac2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,455 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1683302048454"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302048454"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302048454"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302048454"}]},"ts":"1683302048454"} 2023-05-05 15:54:08,457 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-05-05 15:54:08,458 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure df08d89907b6ac6a28b72ec117993ac2, server=jenkins-hbase4.apache.org,40991,1683302023016 in 174 msec 2023-05-05 15:54:08,459 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-05-05 15:54:08,459 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=df08d89907b6ac6a28b72ec117993ac2, ASSIGN in 332 msec 2023-05-05 15:54:08,460 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-05 15:54:08,460 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302048460"}]},"ts":"1683302048460"} 2023-05-05 15:54:08,461 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-05-05 15:54:08,463 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-05-05 15:54:08,465 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 381 msec 2023-05-05 15:54:08,592 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:54:08,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,594 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-05 15:54:08,594 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:08,594 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-05 15:54:08,594 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-05 15:54:08,594 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,594 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,596 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:08,596 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:08,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:08,596 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:08,596 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:08,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:08,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:08,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:08,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:08,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,596 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,597 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:54:08,597 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,597 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,597 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:08,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,597 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:08,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,597 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,597 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:08,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,597 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:08,598 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:08,597 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,597 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,598 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:08,598 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:08,598 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:08,598 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:08,598 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:08,598 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:08,598 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,598 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:08,599 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:08,599 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:54:08,599 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:54:08,599 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:08,599 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. started... 2023-05-05 15:54:08,599 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:08,599 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:08,599 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:08,600 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing df08d89907b6ac6a28b72ec117993ac2 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-05 15:54:08,600 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,602 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,602 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,602 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:08,602 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,602 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:08,615 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf1/12a21e072828498bb83e9dffaca389ad 2023-05-05 15:54:08,650 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf2/dd4d733d767c4c68bdcb726c005625f4 2023-05-05 15:54:08,655 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf1/12a21e072828498bb83e9dffaca389ad as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/12a21e072828498bb83e9dffaca389ad 2023-05-05 15:54:08,660 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/12a21e072828498bb83e9dffaca389ad, entries=2, sequenceid=6, filesize=4.8 K 2023-05-05 15:54:08,660 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf2/dd4d733d767c4c68bdcb726c005625f4 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/dd4d733d767c4c68bdcb726c005625f4 2023-05-05 15:54:08,665 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/dd4d733d767c4c68bdcb726c005625f4, entries=2, sequenceid=6, filesize=4.8 K 2023-05-05 15:54:08,666 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for df08d89907b6ac6a28b72ec117993ac2 in 66ms, sequenceid=6, compaction requested=false 2023-05-05 15:54:08,666 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-05-05 15:54:08,666 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for df08d89907b6ac6a28b72ec117993ac2: 2023-05-05 15:54:08,666 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:08,666 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:54:08,666 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:54:08,666 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,666 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:08,666 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:08,668 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,668 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:08,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:08,668 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,668 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:08,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:08,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:08,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:08,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:08,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:08,670 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-05 15:54:08,670 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@62f8ba80[Count = 0] remaining members to acquire global barrier 2023-05-05 15:54:08,670 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-05 15:54:08,670 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,671 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,671 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,671 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,672 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:08,672 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:08,672 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:08,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,672 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,672 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:08,672 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:54:08,672 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:08,672 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:08,672 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:54:08,672 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:54:08,672 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:54:08,674 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,674 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:08,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:08,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:08,675 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:08,675 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:08,675 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:08,675 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:08,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:08,675 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:08,675 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:08,675 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:08,676 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:08,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:08,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:08,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:08,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:08,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:08,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:08,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:54:08,678 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-05 15:54:08,678 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-05 15:54:08,678 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:54:08,678 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:54:08,678 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,678 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:54:08,680 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,680 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,680 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,680 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:08,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,680 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:08,680 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,681 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:08,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:08,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:08,681 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:08,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:08,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:08,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:08,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:08,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:08,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:08,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,681 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:08,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:08,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,682 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:08,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:08,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:08,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,684 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:08,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:08,684 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,684 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:08,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:08,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:08,696 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,696 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:08,696 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:08,696 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:08,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:08,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:08,696 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:08,697 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-05 15:54:08,697 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:54:08,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:08,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:08,696 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:08,696 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:08,696 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:08,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:08,697 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:08,697 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-05 15:54:08,697 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:08,697 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:54:08,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:08,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:08,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:08,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:08,697 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:08,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:08,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:08,698 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,698 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:08,698 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,698 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,698 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:08,698 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,698 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,797 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:54:08,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:54:08,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:54:08,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-05 15:54:08,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,814 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-05 15:54:08,814 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:08,814 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-05 15:54:08,814 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-05 15:54:08,815 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,815 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,816 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,816 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:08,816 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:08,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:08,817 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:08,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:08,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:08,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:08,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:08,817 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,817 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:54:08,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:08,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,817 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,817 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:08,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:08,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,818 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,818 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,818 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,818 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-05 15:54:08,818 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:08,818 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:08,818 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-05 15:54:08,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,818 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,818 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:08,819 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:08,819 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:08,818 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,819 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:08,819 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:08,819 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:08,819 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:08,819 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-05 15:54:08,819 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,819 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:08,819 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:54:08,819 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:54:08,819 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:08,819 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:08,819 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. started... 2023-05-05 15:54:08,819 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:08,819 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:08,819 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:08,819 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing df08d89907b6ac6a28b72ec117993ac2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:54:08,820 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,821 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,821 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:08,822 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,822 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,822 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:08,830 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf1/434afe9213bb430e9cb8198b4f2222f3 2023-05-05 15:54:08,846 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf2/119b6673ef83418c87eba55400a3fa50 2023-05-05 15:54:08,850 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf1/434afe9213bb430e9cb8198b4f2222f3 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/434afe9213bb430e9cb8198b4f2222f3 2023-05-05 15:54:08,854 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/434afe9213bb430e9cb8198b4f2222f3, entries=10, sequenceid=19, filesize=5.0 K 2023-05-05 15:54:08,855 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf2/119b6673ef83418c87eba55400a3fa50 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/119b6673ef83418c87eba55400a3fa50 2023-05-05 15:54:08,858 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/119b6673ef83418c87eba55400a3fa50, entries=10, sequenceid=19, filesize=5.0 K 2023-05-05 15:54:08,859 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for df08d89907b6ac6a28b72ec117993ac2 in 40ms, sequenceid=19, compaction requested=false 2023-05-05 15:54:08,859 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for df08d89907b6ac6a28b72ec117993ac2: 2023-05-05 15:54:08,859 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:08,859 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:54:08,859 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:54:08,859 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,859 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:08,859 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:08,862 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,862 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:08,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:08,862 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,862 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:08,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:08,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:08,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:08,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:08,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:08,864 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-05 15:54:08,864 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e6595c[Count = 0] remaining members to acquire global barrier 2023-05-05 15:54:08,864 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-05 15:54:08,864 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,865 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,865 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,865 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,866 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:08,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,866 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:08,866 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,866 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:54:08,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,866 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:54:08,866 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:08,866 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:08,866 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:08,866 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:54:08,866 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:08,866 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:54:08,867 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,867 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:08,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:08,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:08,867 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:08,868 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:08,868 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:08,868 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:08,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:08,868 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:08,868 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:08,868 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:08,868 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:08,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:08,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:08,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:08,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:08,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:08,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:08,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:54:08,871 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-05 15:54:08,871 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-05 15:54:08,871 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:54:08,871 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:54:08,871 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,871 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:54:08,874 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,874 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,874 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:08,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:08,874 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,874 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:08,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,874 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:08,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,875 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:08,874 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:08,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:08,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:08,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:08,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:08,875 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:08,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:08,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:08,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:08,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,876 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:08,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:08,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:08,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:08,877 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:08,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,877 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:08,878 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:08,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:08,880 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:08,880 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,880 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:08,880 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:08,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:08,880 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:08,880 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:08,880 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-05 15:54:08,881 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:54:08,880 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:08,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:08,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:08,880 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:08,881 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:08,881 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,881 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:08,881 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,881 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,881 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:08,881 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-05 15:54:08,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:08,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:08,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:08,881 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:08,881 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:08,881 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:08,881 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:08,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:08,881 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:54:08,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:08,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:08,982 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:54:08,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:54:08,995 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:54:08,997 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-05 15:54:08,997 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,997 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-05 15:54:08,997 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:08,997 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-05 15:54:08,997 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-05 15:54:08,997 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,998 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:08,999 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,999 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:08,999 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:08,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:08,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:08,999 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:08,999 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:08,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:08,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:08,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:08,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:08,999 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:54:08,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,000 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,000 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,000 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:09,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,000 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:09,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,000 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-05 15:54:09,000 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,000 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,000 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:09,000 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:09,001 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:09,001 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-05 15:54:09,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:09,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,001 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:09,001 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,001 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,001 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:09,001 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:09,001 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:09,001 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-05 15:54:09,001 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:09,001 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,001 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:09,001 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:09,001 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:09,001 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:09,001 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:09,001 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:54:09,001 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:54:09,001 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:09,001 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. started... 2023-05-05 15:54:09,001 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing df08d89907b6ac6a28b72ec117993ac2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:54:09,002 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,004 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,004 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,004 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:09,004 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,004 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:09,015 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf1/8593999d3ff54150bdb2dd76b2f46d3f 2023-05-05 15:54:09,030 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf2/3d64d04b356c462c80aeb3d7341d16b5 2023-05-05 15:54:09,034 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf1/8593999d3ff54150bdb2dd76b2f46d3f as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/8593999d3ff54150bdb2dd76b2f46d3f 2023-05-05 15:54:09,038 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/8593999d3ff54150bdb2dd76b2f46d3f, entries=10, sequenceid=32, filesize=5.0 K 2023-05-05 15:54:09,039 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf2/3d64d04b356c462c80aeb3d7341d16b5 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/3d64d04b356c462c80aeb3d7341d16b5 2023-05-05 15:54:09,044 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/3d64d04b356c462c80aeb3d7341d16b5, entries=10, sequenceid=32, filesize=5.0 K 2023-05-05 15:54:09,045 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for df08d89907b6ac6a28b72ec117993ac2 in 44ms, sequenceid=32, compaction requested=true 2023-05-05 15:54:09,045 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for df08d89907b6ac6a28b72ec117993ac2: 2023-05-05 15:54:09,045 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:09,045 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:54:09,045 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:54:09,045 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,045 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:09,045 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:09,050 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,050 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:09,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:09,050 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,050 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:09,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:09,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:09,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:09,052 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-05 15:54:09,052 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@73a2483f[Count = 0] remaining members to acquire global barrier 2023-05-05 15:54:09,052 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-05 15:54:09,052 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,053 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,053 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,053 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,054 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:09,054 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:09,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,054 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:54:09,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,054 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,054 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:54:09,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,054 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:09,054 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:09,054 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:09,054 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:54:09,054 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:09,054 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:54:09,061 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:09,061 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:09,061 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:09,062 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:09,063 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:09,063 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,063 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:09,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:09,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:09,063 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:09,063 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:09,063 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:09,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:09,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:09,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:09,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:54:09,066 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-05 15:54:09,066 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-05 15:54:09,066 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:54:09,066 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:54:09,066 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,066 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:54:09,067 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,067 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,067 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,067 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,067 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,068 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,067 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:09,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:09,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,067 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,068 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:09,068 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:09,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,070 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:09,071 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,071 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,073 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,073 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,073 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,073 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,073 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,073 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:09,073 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,073 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-05 15:54:09,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,073 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,074 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:54:09,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,073 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,074 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,074 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,074 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,074 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-05 15:54:09,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,074 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,074 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:54:09,074 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,074 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,075 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,075 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,075 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,174 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:54:09,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:54:09,189 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:54:09,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-05 15:54:09,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,190 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-05 15:54:09,190 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:09,190 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-05 15:54:09,190 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-05 15:54:09,191 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,191 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,193 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,193 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,193 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,193 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,194 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,194 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:54:09,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,194 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,194 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,194 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:09,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:09,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:09,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,195 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,195 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,195 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,195 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-05 15:54:09,195 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,195 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-05 15:54:09,195 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:09,195 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,195 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:09,195 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:09,195 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:09,195 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:09,195 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:09,195 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:09,196 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:09,197 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,197 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,197 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:09,200 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-05 15:54:09,200 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:54:09,200 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:54:09,200 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,200 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:09,200 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:09,200 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. started... 2023-05-05 15:54:09,200 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:09,200 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:09,200 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:09,200 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:09,200 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing df08d89907b6ac6a28b72ec117993ac2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:54:09,201 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,202 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,202 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:09,210 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf1/f87f9f855cfc4676bdef6c17cad64e1c 2023-05-05 15:54:09,225 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf2/5dbf1121ee934ff395d553d9c0733a84 2023-05-05 15:54:09,229 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf1/f87f9f855cfc4676bdef6c17cad64e1c as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/f87f9f855cfc4676bdef6c17cad64e1c 2023-05-05 15:54:09,233 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/f87f9f855cfc4676bdef6c17cad64e1c, entries=10, sequenceid=45, filesize=5.0 K 2023-05-05 15:54:09,234 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf2/5dbf1121ee934ff395d553d9c0733a84 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/5dbf1121ee934ff395d553d9c0733a84 2023-05-05 15:54:09,237 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/5dbf1121ee934ff395d553d9c0733a84, entries=10, sequenceid=45, filesize=5.0 K 2023-05-05 15:54:09,238 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for df08d89907b6ac6a28b72ec117993ac2 in 38ms, sequenceid=45, compaction requested=true 2023-05-05 15:54:09,238 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for df08d89907b6ac6a28b72ec117993ac2: 2023-05-05 15:54:09,238 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:09,239 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:54:09,239 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:54:09,239 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,239 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:09,239 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:09,240 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,240 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:09,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:09,240 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,240 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:09,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:09,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:09,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:09,242 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-05 15:54:09,242 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@758d794c[Count = 0] remaining members to acquire global barrier 2023-05-05 15:54:09,242 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-05 15:54:09,242 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,245 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,245 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,245 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,245 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:09,245 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:09,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,245 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,245 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:09,245 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:54:09,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,245 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:09,245 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:09,245 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:54:09,245 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:54:09,245 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:09,246 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:54:09,247 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:09,247 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,247 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:09,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:09,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:09,247 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:09,247 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:09,247 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:09,247 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:09,248 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:09,248 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:09,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:09,248 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:09,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:09,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:09,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:54:09,251 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-05 15:54:09,251 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-05 15:54:09,251 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:54:09,251 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:54:09,251 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,251 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:54:09,257 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,257 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,257 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,257 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,257 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,257 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,257 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:09,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:09,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,258 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:09,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,259 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,259 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:09,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:09,261 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,261 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,261 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,265 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:09,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,265 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-05 15:54:09,265 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:54:09,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,265 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,265 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,265 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,265 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,265 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,265 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,265 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,265 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,265 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,265 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,265 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-05 15:54:09,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,265 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:54:09,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,366 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:54:09,367 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:54:09,379 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:54:09,380 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-05 15:54:09,380 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,380 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-05 15:54:09,381 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:09,381 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-05 15:54:09,381 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-05 15:54:09,381 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,381 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,382 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,382 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,382 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,382 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,382 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,382 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,383 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,383 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:54:09,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,383 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,383 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:09,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,384 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,384 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:09,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,384 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,384 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-05 15:54:09,384 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,384 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:09,384 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-05 15:54:09,384 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:09,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:09,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,384 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,385 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,385 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:09,385 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:09,384 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:09,385 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:09,385 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:09,385 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:09,385 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-05 15:54:09,385 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:09,385 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:09,385 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,385 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:09,385 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:09,385 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:09,385 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:54:09,385 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:54:09,385 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:09,385 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. started... 2023-05-05 15:54:09,386 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing df08d89907b6ac6a28b72ec117993ac2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:54:09,386 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,388 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,388 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,388 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:09,388 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,388 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:09,400 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf1/de1a5ea53f2a42578b0dcd4c71037963 2023-05-05 15:54:09,414 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf2/5dfe0faf8f924ef29b2ab0594e62a7bc 2023-05-05 15:54:09,419 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf1/de1a5ea53f2a42578b0dcd4c71037963 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/de1a5ea53f2a42578b0dcd4c71037963 2023-05-05 15:54:09,423 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/de1a5ea53f2a42578b0dcd4c71037963, entries=10, sequenceid=58, filesize=5.0 K 2023-05-05 15:54:09,423 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf2/5dfe0faf8f924ef29b2ab0594e62a7bc as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/5dfe0faf8f924ef29b2ab0594e62a7bc 2023-05-05 15:54:09,427 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/5dfe0faf8f924ef29b2ab0594e62a7bc, entries=10, sequenceid=58, filesize=5.0 K 2023-05-05 15:54:09,428 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for df08d89907b6ac6a28b72ec117993ac2 in 43ms, sequenceid=58, compaction requested=true 2023-05-05 15:54:09,428 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for df08d89907b6ac6a28b72ec117993ac2: 2023-05-05 15:54:09,428 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:09,428 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:54:09,428 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:54:09,428 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,428 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:09,428 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:09,430 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,430 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:09,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:09,430 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,430 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:09,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:09,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:09,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:09,432 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-05 15:54:09,432 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5206f1c7[Count = 0] remaining members to acquire global barrier 2023-05-05 15:54:09,432 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-05 15:54:09,432 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,433 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,433 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,434 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,434 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,434 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:54:09,434 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:09,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,434 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:09,434 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:09,434 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:09,434 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:09,434 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:54:09,434 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:09,434 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:54:09,434 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:54:09,435 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,435 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:09,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,436 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:09,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:09,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:09,436 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:09,438 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:09,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:09,438 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:09,438 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:09,438 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:09,438 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:09,438 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:09,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:09,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:09,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:54:09,441 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-05 15:54:09,441 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-05 15:54:09,441 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:54:09,441 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:54:09,441 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,441 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:54:09,442 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,442 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,442 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,443 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:09,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:09,442 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,442 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,443 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:09,443 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,443 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:09,444 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:09,446 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,446 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,447 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,450 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,450 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,450 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,450 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,450 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,450 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,450 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-05 15:54:09,450 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:54:09,450 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,450 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,450 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:09,450 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,450 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-05 15:54:09,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,451 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:54:09,450 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,451 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,451 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,451 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,451 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,451 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,451 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,451 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,551 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:54:09,552 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:54:09,565 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:54:09,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-05 15:54:09,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,566 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-05 15:54:09,566 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:09,566 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-05 15:54:09,566 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-05 15:54:09,567 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,567 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,575 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,575 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,575 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,575 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,576 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,576 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:54:09,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,576 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:09,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,577 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,577 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,577 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,577 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-05 15:54:09,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:09,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:09,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,577 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,577 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,577 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:09,577 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:09,577 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,577 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-05 15:54:09,577 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,577 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:09,578 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:09,578 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:09,578 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:09,578 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:09,578 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:09,578 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-05 15:54:09,578 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:09,578 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,578 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:09,578 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:09,578 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:09,578 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:09,578 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:54:09,578 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:54:09,578 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:09,578 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. started... 2023-05-05 15:54:09,579 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing df08d89907b6ac6a28b72ec117993ac2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:54:09,581 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,581 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,582 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,582 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:09,582 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,582 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:09,592 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf1/e68abacff98449d58bd61ca4ffe85094 2023-05-05 15:54:09,608 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf2/f775112a6fc94a92b672ce0af66e793b 2023-05-05 15:54:09,612 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf1/e68abacff98449d58bd61ca4ffe85094 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/e68abacff98449d58bd61ca4ffe85094 2023-05-05 15:54:09,616 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/e68abacff98449d58bd61ca4ffe85094, entries=10, sequenceid=71, filesize=5.0 K 2023-05-05 15:54:09,617 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf2/f775112a6fc94a92b672ce0af66e793b as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/f775112a6fc94a92b672ce0af66e793b 2023-05-05 15:54:09,621 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/f775112a6fc94a92b672ce0af66e793b, entries=10, sequenceid=71, filesize=5.0 K 2023-05-05 15:54:09,621 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for df08d89907b6ac6a28b72ec117993ac2 in 42ms, sequenceid=71, compaction requested=true 2023-05-05 15:54:09,621 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for df08d89907b6ac6a28b72ec117993ac2: 2023-05-05 15:54:09,621 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:09,622 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:54:09,622 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:54:09,622 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,622 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:09,622 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:09,623 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,623 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:09,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:09,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:09,624 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,624 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:09,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:09,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:09,626 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-05 15:54:09,626 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@669e6e62[Count = 0] remaining members to acquire global barrier 2023-05-05 15:54:09,626 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-05 15:54:09,626 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,628 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,628 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,628 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,628 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:09,629 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:09,629 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:09,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,629 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,629 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:09,629 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:54:09,629 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:54:09,629 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:09,629 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:54:09,629 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:09,629 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:54:09,630 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:09,630 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:09,630 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:09,630 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:09,630 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:09,630 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:09,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:09,630 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:09,630 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:09,631 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:09,631 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:09,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:09,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:09,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:09,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:54:09,634 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-05 15:54:09,634 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-05 15:54:09,634 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:54:09,634 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:54:09,634 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,634 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:54:09,635 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,636 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,635 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,635 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:09,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:09,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,636 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,636 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,636 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,636 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:09,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,637 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,637 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:09,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:09,638 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,639 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,639 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,642 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,642 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,642 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,643 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,643 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,643 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,643 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-05 15:54:09,643 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:54:09,643 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:09,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,643 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,643 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,643 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,643 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,643 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-05 15:54:09,643 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,643 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,644 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,644 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:54:09,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,644 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,644 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,644 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,644 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,744 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:54:09,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:54:09,760 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:54:09,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-05 15:54:09,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,762 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-05 15:54:09,762 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:09,762 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-05 15:54:09,762 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-05 15:54:09,762 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,762 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,765 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,765 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,765 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,765 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,765 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,765 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:54:09,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,766 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,766 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,766 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:09,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:09,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:09,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,766 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,766 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,766 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,767 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-05 15:54:09,767 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,767 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:09,767 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:09,767 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-05 15:54:09,767 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:09,767 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,767 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:09,767 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:09,767 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:09,767 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:09,767 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:09,767 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:09,767 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:09,767 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-05 15:54:09,767 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,767 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:09,767 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:09,767 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:09,768 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:54:09,768 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:54:09,768 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:09,768 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. started... 2023-05-05 15:54:09,768 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing df08d89907b6ac6a28b72ec117993ac2 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-05-05 15:54:09,768 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,769 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,769 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:09,769 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,769 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,769 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:09,778 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf1/5d5c454854a3429ea5e9adedb86c8276 2023-05-05 15:54:09,782 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5d5c454854a3429ea5e9adedb86c8276 2023-05-05 15:54:09,792 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf2/9ae30df8bc6a4dc198df8a5bfb0a5075 2023-05-05 15:54:09,795 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9ae30df8bc6a4dc198df8a5bfb0a5075 2023-05-05 15:54:09,796 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf1/5d5c454854a3429ea5e9adedb86c8276 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/5d5c454854a3429ea5e9adedb86c8276 2023-05-05 15:54:09,799 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5d5c454854a3429ea5e9adedb86c8276 2023-05-05 15:54:09,799 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/5d5c454854a3429ea5e9adedb86c8276, entries=9, sequenceid=83, filesize=5.1 K 2023-05-05 15:54:09,800 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf2/9ae30df8bc6a4dc198df8a5bfb0a5075 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/9ae30df8bc6a4dc198df8a5bfb0a5075 2023-05-05 15:54:09,804 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9ae30df8bc6a4dc198df8a5bfb0a5075 2023-05-05 15:54:09,804 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/9ae30df8bc6a4dc198df8a5bfb0a5075, entries=9, sequenceid=83, filesize=5.1 K 2023-05-05 15:54:09,805 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for df08d89907b6ac6a28b72ec117993ac2 in 37ms, sequenceid=83, compaction requested=true 2023-05-05 15:54:09,805 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for df08d89907b6ac6a28b72ec117993ac2: 2023-05-05 15:54:09,805 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:09,805 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:54:09,805 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:54:09,805 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,805 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:09,805 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:09,807 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,807 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:09,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:09,807 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:09,807 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:09,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:09,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:09,809 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-05 15:54:09,809 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79c213e[Count = 0] remaining members to acquire global barrier 2023-05-05 15:54:09,809 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-05 15:54:09,809 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,811 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,811 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,811 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,812 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,812 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:54:09,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,812 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:09,812 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:09,812 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:09,812 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:09,812 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:09,812 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:54:09,812 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:54:09,812 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:09,812 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:54:09,819 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:09,819 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:09,819 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,819 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:09,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:09,819 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:09,819 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:09,819 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:09,819 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:09,819 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:09,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:09,819 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:09,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:09,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:09,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:09,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:54:09,822 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-05 15:54:09,822 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-05 15:54:09,822 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:54:09,822 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:54:09,822 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,823 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:54:09,824 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,824 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,824 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:09,824 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:09,824 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,824 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,824 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,825 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:09,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,825 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,825 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:09,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,827 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:09,827 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,827 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,831 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,831 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,831 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,831 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,831 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,831 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-05 15:54:09,831 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:54:09,831 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,831 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,831 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,831 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,831 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:09,831 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,831 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-05 15:54:09,831 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,832 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,831 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:54:09,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,832 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,832 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,832 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,832 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,832 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,932 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:54:09,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:54:09,941 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:54:09,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-05 15:54:09,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,942 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-05 15:54:09,942 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:09,942 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-05 15:54:09,943 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-05 15:54:09,943 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,943 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,944 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,944 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,944 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,944 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:09,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,944 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,944 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:54:09,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:09,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:09,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,945 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,945 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,945 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:09,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,945 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:09,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,945 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:09,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,946 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,946 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-05 15:54:09,946 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,946 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-05 15:54:09,946 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:09,946 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-05 15:54:09,946 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,946 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:09,946 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:09,946 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:09,946 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,946 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:09,946 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:09,946 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:09,946 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:09,946 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:09,946 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:09,946 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:09,946 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:09,947 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:09,947 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:54:09,947 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:54:09,947 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:09,947 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. started... 2023-05-05 15:54:09,947 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing df08d89907b6ac6a28b72ec117993ac2 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-05-05 15:54:09,953 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,953 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,954 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,954 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:09,954 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,954 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:09,960 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf1/a0e50c325aef45bd913cb3678df787a7 2023-05-05 15:54:09,964 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a0e50c325aef45bd913cb3678df787a7 2023-05-05 15:54:09,971 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf2/c31b04df32c4455aab59ebd1cada05ae 2023-05-05 15:54:09,974 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c31b04df32c4455aab59ebd1cada05ae 2023-05-05 15:54:09,975 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf1/a0e50c325aef45bd913cb3678df787a7 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/a0e50c325aef45bd913cb3678df787a7 2023-05-05 15:54:09,978 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a0e50c325aef45bd913cb3678df787a7 2023-05-05 15:54:09,979 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/a0e50c325aef45bd913cb3678df787a7, entries=5, sequenceid=91, filesize=5.0 K 2023-05-05 15:54:09,979 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf2/c31b04df32c4455aab59ebd1cada05ae as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/c31b04df32c4455aab59ebd1cada05ae 2023-05-05 15:54:09,983 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c31b04df32c4455aab59ebd1cada05ae 2023-05-05 15:54:09,983 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/c31b04df32c4455aab59ebd1cada05ae, entries=5, sequenceid=91, filesize=5.0 K 2023-05-05 15:54:09,983 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for df08d89907b6ac6a28b72ec117993ac2 in 36ms, sequenceid=91, compaction requested=true 2023-05-05 15:54:09,984 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for df08d89907b6ac6a28b72ec117993ac2: 2023-05-05 15:54:09,984 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:09,984 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:54:09,984 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:54:09,984 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,984 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:09,984 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:09,985 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,985 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:09,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:09,986 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,986 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:09,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:09,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:09,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:09,987 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-05 15:54:09,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@19457f98[Count = 0] remaining members to acquire global barrier 2023-05-05 15:54:09,988 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-05 15:54:09,988 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,990 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,990 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,990 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,990 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,990 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:09,990 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:09,990 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:09,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,990 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:54:09,990 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:09,990 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:09,990 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:54:09,990 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:09,990 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:54:09,990 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:54:09,991 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:09,991 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:09,991 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:09,992 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:09,992 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:09,992 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,992 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:09,992 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:09,992 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:09,992 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:09,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:09,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:09,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:09,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:09,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:09,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:54:09,995 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-05 15:54:09,995 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-05 15:54:09,996 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:54:09,996 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:54:09,996 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,996 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:54:09,997 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,997 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,997 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,997 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,997 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,997 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,997 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:09,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:09,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:09,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:09,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:09,998 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:09,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:09,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,998 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:09,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,999 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:09,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:09,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:09,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:10,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:10,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:10,000 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:10,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,000 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:10,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:10,001 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:10,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,007 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,007 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,007 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:10,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:10,007 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:10,007 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:10,007 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:10,007 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:10,007 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:10,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:10,007 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-05 15:54:10,008 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:54:10,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:10,007 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:10,007 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:10,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:10,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:10,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:10,007 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,007 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:10,008 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-05 15:54:10,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:10,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:10,008 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:10,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:10,008 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:54:10,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:10,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:10,008 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,009 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:10,009 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:10,009 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,009 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,108 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:54:10,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:54:10,118 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:54:10,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-05 15:54:10,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,119 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-05 15:54:10,119 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:10,119 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-05 15:54:10,119 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-05 15:54:10,120 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,120 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,121 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:10,121 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:10,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:10,121 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:10,121 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:10,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:10,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:10,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:10,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:10,122 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,122 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:54:10,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,122 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,122 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,122 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:10,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,122 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:10,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,122 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:54:10,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,123 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,123 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-05 15:54:10,123 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,123 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-05 15:54:10,123 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,123 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-05 15:54:10,123 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:10,123 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,123 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:10,123 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-05 15:54:10,123 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:10,123 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:10,123 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:10,123 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:54:10,123 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:10,123 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:10,123 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:54:10,123 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:54:10,123 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-05 15:54:10,124 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:10,123 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:10,123 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:10,124 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. started... 2023-05-05 15:54:10,124 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:10,124 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing df08d89907b6ac6a28b72ec117993ac2 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-05-05 15:54:10,125 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,125 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,125 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,125 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,125 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:10,125 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:10,133 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf1/5e04725445fd4322809d1a9d85503ec1 2023-05-05 15:54:10,136 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5e04725445fd4322809d1a9d85503ec1 2023-05-05 15:54:10,144 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf2/f1c90388d6a64950a4d581e92c36d16a 2023-05-05 15:54:10,147 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f1c90388d6a64950a4d581e92c36d16a 2023-05-05 15:54:10,148 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf1/5e04725445fd4322809d1a9d85503ec1 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/5e04725445fd4322809d1a9d85503ec1 2023-05-05 15:54:10,152 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5e04725445fd4322809d1a9d85503ec1 2023-05-05 15:54:10,152 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/5e04725445fd4322809d1a9d85503ec1, entries=5, sequenceid=99, filesize=5.0 K 2023-05-05 15:54:10,152 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf2/f1c90388d6a64950a4d581e92c36d16a as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/f1c90388d6a64950a4d581e92c36d16a 2023-05-05 15:54:10,156 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f1c90388d6a64950a4d581e92c36d16a 2023-05-05 15:54:10,156 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/f1c90388d6a64950a4d581e92c36d16a, entries=5, sequenceid=99, filesize=5.0 K 2023-05-05 15:54:10,157 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for df08d89907b6ac6a28b72ec117993ac2 in 33ms, sequenceid=99, compaction requested=true 2023-05-05 15:54:10,157 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for df08d89907b6ac6a28b72ec117993ac2: 2023-05-05 15:54:10,157 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:10,157 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:54:10,157 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:54:10,157 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,157 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-05 15:54:10,157 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-05 15:54:10,160 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,160 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:10,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:10,160 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,160 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:54:10,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:10,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:10,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:10,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:10,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:10,162 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-05 15:54:10,162 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ca82dbe[Count = 0] remaining members to acquire global barrier 2023-05-05 15:54:10,162 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-05 15:54:10,162 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,163 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,163 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,163 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,164 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:10,164 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:10,164 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,164 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:54:10,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,164 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:54:10,164 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:10,164 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:10,164 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-05 15:54:10,164 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-05 15:54:10,164 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:54:10,164 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:54:10,165 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:10,165 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:10,165 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:10,166 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:10,166 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,166 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:10,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,166 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:10,166 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-05 15:54:10,166 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:10,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:10,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:10,166 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-05 15:54:10,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:10,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:10,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:10,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:10,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:10,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:10,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:10,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:54:10,169 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-05 15:54:10,169 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-05 15:54:10,169 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:54:10,169 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:54:10,169 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,169 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:54:10,172 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,172 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,172 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,172 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:54:10,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:54:10,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,172 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:10,172 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:10,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,172 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:10,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:54:10,172 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:10,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:10,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:10,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:10,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:10,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:10,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:10,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,173 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:10,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:54:10,173 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:10,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:10,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:54:10,175 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:10,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,176 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:10,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:10,176 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:10,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,178 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,178 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,178 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:10,178 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:10,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:10,178 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:54:10,178 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:10,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:10,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:10,178 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:10,179 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-05 15:54:10,179 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:10,178 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:10,178 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:54:10,178 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:54:10,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:10,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:54:10,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:10,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:10,179 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,179 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:54:10,179 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,179 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:10,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:10,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:54:10,179 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:54:10,179 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:10,179 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-05 15:54:10,179 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,180 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:54:10,180 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:10,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:54:10,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:54:10,280 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:54:10,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:54:10,283 DEBUG [Listener at localhost/35117] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,284 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=df08d89907b6ac6a28b72ec117993ac2, daughterA=6cb2bd681dce908bd0c281b2c71d45e8, daughterB=2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:54:10,285 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=df08d89907b6ac6a28b72ec117993ac2, daughterA=6cb2bd681dce908bd0c281b2c71d45e8, daughterB=2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:54:10,285 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=df08d89907b6ac6a28b72ec117993ac2, daughterA=6cb2bd681dce908bd0c281b2c71d45e8, daughterB=2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:54:10,285 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=df08d89907b6ac6a28b72ec117993ac2, daughterA=6cb2bd681dce908bd0c281b2c71d45e8, daughterB=2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:54:10,287 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=df08d89907b6ac6a28b72ec117993ac2, UNASSIGN}] 2023-05-05 15:54:10,288 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=df08d89907b6ac6a28b72ec117993ac2, UNASSIGN 2023-05-05 15:54:10,288 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=df08d89907b6ac6a28b72ec117993ac2, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,289 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1683302050288"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302050288"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302050288"}]},"ts":"1683302050288"} 2023-05-05 15:54:10,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure df08d89907b6ac6a28b72ec117993ac2, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:54:10,441 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,441 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing df08d89907b6ac6a28b72ec117993ac2, disabling compactions & flushes 2023-05-05 15:54:10,441 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:10,441 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:10,441 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. after waiting 0 ms 2023-05-05 15:54:10,441 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:10,441 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing df08d89907b6ac6a28b72ec117993ac2 2/2 column families, dataSize=50 B heapSize=720 B 2023-05-05 15:54:10,451 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf1/ab3a9bf885de4438ad1f504ac52c62c2 2023-05-05 15:54:10,455 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ab3a9bf885de4438ad1f504ac52c62c2 2023-05-05 15:54:10,463 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf2/e7bd85555d934280bda43b21281bb981 2023-05-05 15:54:10,466 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e7bd85555d934280bda43b21281bb981 2023-05-05 15:54:10,467 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf1/ab3a9bf885de4438ad1f504ac52c62c2 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/ab3a9bf885de4438ad1f504ac52c62c2 2023-05-05 15:54:10,470 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ab3a9bf885de4438ad1f504ac52c62c2 2023-05-05 15:54:10,471 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/ab3a9bf885de4438ad1f504ac52c62c2, entries=1, sequenceid=103, filesize=4.9 K 2023-05-05 15:54:10,471 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.tmp/cf2/e7bd85555d934280bda43b21281bb981 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/e7bd85555d934280bda43b21281bb981 2023-05-05 15:54:10,475 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e7bd85555d934280bda43b21281bb981 2023-05-05 15:54:10,475 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/e7bd85555d934280bda43b21281bb981, entries=1, sequenceid=103, filesize=4.9 K 2023-05-05 15:54:10,476 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for df08d89907b6ac6a28b72ec117993ac2 in 35ms, sequenceid=103, compaction requested=true 2023-05-05 15:54:10,480 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-05-05 15:54:10,481 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:54:10,481 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2. 2023-05-05 15:54:10,481 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for df08d89907b6ac6a28b72ec117993ac2: 2023-05-05 15:54:10,482 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,483 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=df08d89907b6ac6a28b72ec117993ac2, regionState=CLOSED 2023-05-05 15:54:10,483 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1683302050482"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302050482"}]},"ts":"1683302050482"} 2023-05-05 15:54:10,485 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-05-05 15:54:10,485 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure df08d89907b6ac6a28b72ec117993ac2, server=jenkins-hbase4.apache.org,40991,1683302023016 in 194 msec 2023-05-05 15:54:10,487 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-05-05 15:54:10,487 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=df08d89907b6ac6a28b72ec117993ac2, UNASSIGN in 198 msec 2023-05-05 15:54:10,501 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=df08d89907b6ac6a28b72ec117993ac2, threads=16 2023-05-05 15:54:10,501 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/119b6673ef83418c87eba55400a3fa50 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,501 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/3d64d04b356c462c80aeb3d7341d16b5 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,501 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/5dbf1121ee934ff395d553d9c0733a84 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,509 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/5dfe0faf8f924ef29b2ab0594e62a7bc for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,509 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/9ae30df8bc6a4dc198df8a5bfb0a5075 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,509 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/c31b04df32c4455aab59ebd1cada05ae for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,509 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/dd4d733d767c4c68bdcb726c005625f4 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,509 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/e7bd85555d934280bda43b21281bb981 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,509 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/f1c90388d6a64950a4d581e92c36d16a for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,510 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/f775112a6fc94a92b672ce0af66e793b for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,510 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/12a21e072828498bb83e9dffaca389ad for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,512 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/434afe9213bb430e9cb8198b4f2222f3 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,512 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/5d5c454854a3429ea5e9adedb86c8276 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,515 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/5e04725445fd4322809d1a9d85503ec1 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,515 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/8593999d3ff54150bdb2dd76b2f46d3f for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,516 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/a0e50c325aef45bd913cb3678df787a7 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,519 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f1c90388d6a64950a4d581e92c36d16a 2023-05-05 15:54:10,527 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/5dbf1121ee934ff395d553d9c0733a84, top=false 2023-05-05 15:54:10,528 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9ae30df8bc6a4dc198df8a5bfb0a5075 2023-05-05 15:54:10,531 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f1c90388d6a64950a4d581e92c36d16a 2023-05-05 15:54:10,531 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/f1c90388d6a64950a4d581e92c36d16a, top=true 2023-05-05 15:54:10,533 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/f775112a6fc94a92b672ce0af66e793b, top=false 2023-05-05 15:54:10,534 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5e04725445fd4322809d1a9d85503ec1 2023-05-05 15:54:10,534 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5d5c454854a3429ea5e9adedb86c8276 2023-05-05 15:54:10,535 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/5dfe0faf8f924ef29b2ab0594e62a7bc, top=false 2023-05-05 15:54:10,535 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e7bd85555d934280bda43b21281bb981 2023-05-05 15:54:10,536 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c31b04df32c4455aab59ebd1cada05ae 2023-05-05 15:54:10,538 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a0e50c325aef45bd913cb3678df787a7 2023-05-05 15:54:10,539 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5e04725445fd4322809d1a9d85503ec1 2023-05-05 15:54:10,539 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/5e04725445fd4322809d1a9d85503ec1, top=true 2023-05-05 15:54:10,544 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c31b04df32c4455aab59ebd1cada05ae 2023-05-05 15:54:10,544 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e7bd85555d934280bda43b21281bb981 2023-05-05 15:54:10,545 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/e7bd85555d934280bda43b21281bb981, top=true 2023-05-05 15:54:10,545 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/c31b04df32c4455aab59ebd1cada05ae, top=true 2023-05-05 15:54:10,546 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a0e50c325aef45bd913cb3678df787a7 2023-05-05 15:54:10,546 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/a0e50c325aef45bd913cb3678df787a7, top=true 2023-05-05 15:54:10,616 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.splits/6cb2bd681dce908bd0c281b2c71d45e8/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-5dfe0faf8f924ef29b2ab0594e62a7bc for child: 6cb2bd681dce908bd0c281b2c71d45e8, parent: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,616 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.splits/2d9826f21eb94a559f0d5b68d98cb98d/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-c31b04df32c4455aab59ebd1cada05ae for child: 2d9826f21eb94a559f0d5b68d98cb98d, parent: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,616 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/c31b04df32c4455aab59ebd1cada05ae for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,616 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.splits/2d9826f21eb94a559f0d5b68d98cb98d/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-e7bd85555d934280bda43b21281bb981 for child: 2d9826f21eb94a559f0d5b68d98cb98d, parent: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,616 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/e7bd85555d934280bda43b21281bb981 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,616 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/ab3a9bf885de4438ad1f504ac52c62c2 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,616 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.splits/6cb2bd681dce908bd0c281b2c71d45e8/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-f775112a6fc94a92b672ce0af66e793b for child: 6cb2bd681dce908bd0c281b2c71d45e8, parent: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,616 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/de1a5ea53f2a42578b0dcd4c71037963 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,625 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ab3a9bf885de4438ad1f504ac52c62c2 2023-05-05 15:54:10,625 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/f775112a6fc94a92b672ce0af66e793b for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,625 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/e68abacff98449d58bd61ca4ffe85094 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,626 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/5dfe0faf8f924ef29b2ab0594e62a7bc for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,626 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/f87f9f855cfc4676bdef6c17cad64e1c for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,626 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.splits/2d9826f21eb94a559f0d5b68d98cb98d/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-5e04725445fd4322809d1a9d85503ec1 for child: 2d9826f21eb94a559f0d5b68d98cb98d, parent: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,626 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/5e04725445fd4322809d1a9d85503ec1 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,626 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/de1a5ea53f2a42578b0dcd4c71037963, top=false 2023-05-05 15:54:10,627 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.splits/2d9826f21eb94a559f0d5b68d98cb98d/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-a0e50c325aef45bd913cb3678df787a7 for child: 2d9826f21eb94a559f0d5b68d98cb98d, parent: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,627 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/a0e50c325aef45bd913cb3678df787a7 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,627 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.splits/2d9826f21eb94a559f0d5b68d98cb98d/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-f1c90388d6a64950a4d581e92c36d16a for child: 2d9826f21eb94a559f0d5b68d98cb98d, parent: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,628 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/f1c90388d6a64950a4d581e92c36d16a for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,631 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ab3a9bf885de4438ad1f504ac52c62c2 2023-05-05 15:54:10,631 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/ab3a9bf885de4438ad1f504ac52c62c2, top=true 2023-05-05 15:54:10,638 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.splits/6cb2bd681dce908bd0c281b2c71d45e8/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-5dbf1121ee934ff395d553d9c0733a84 for child: 6cb2bd681dce908bd0c281b2c71d45e8, parent: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,645 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/e68abacff98449d58bd61ca4ffe85094, top=false 2023-05-05 15:54:10,646 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/f87f9f855cfc4676bdef6c17cad64e1c, top=false 2023-05-05 15:54:10,649 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/5dbf1121ee934ff395d553d9c0733a84 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,651 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5d5c454854a3429ea5e9adedb86c8276 2023-05-05 15:54:10,676 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.splits/6cb2bd681dce908bd0c281b2c71d45e8/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-de1a5ea53f2a42578b0dcd4c71037963 for child: 6cb2bd681dce908bd0c281b2c71d45e8, parent: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,684 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/de1a5ea53f2a42578b0dcd4c71037963 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,687 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.splits/2d9826f21eb94a559f0d5b68d98cb98d/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-ab3a9bf885de4438ad1f504ac52c62c2 for child: 2d9826f21eb94a559f0d5b68d98cb98d, parent: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,687 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.splits/6cb2bd681dce908bd0c281b2c71d45e8/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-f87f9f855cfc4676bdef6c17cad64e1c for child: 6cb2bd681dce908bd0c281b2c71d45e8, parent: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,687 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/ab3a9bf885de4438ad1f504ac52c62c2 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,691 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/f87f9f855cfc4676bdef6c17cad64e1c for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,708 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/.splits/6cb2bd681dce908bd0c281b2c71d45e8/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-e68abacff98449d58bd61ca4ffe85094 for child: 6cb2bd681dce908bd0c281b2c71d45e8, parent: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,712 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/e68abacff98449d58bd61ca4ffe85094 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,729 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/8593999d3ff54150bdb2dd76b2f46d3f for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,737 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9ae30df8bc6a4dc198df8a5bfb0a5075 2023-05-05 15:54:10,748 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/119b6673ef83418c87eba55400a3fa50 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,748 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/5d5c454854a3429ea5e9adedb86c8276 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,748 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/3d64d04b356c462c80aeb3d7341d16b5 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,749 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/dd4d733d767c4c68bdcb726c005625f4 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,760 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/434afe9213bb430e9cb8198b4f2222f3 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,764 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/12a21e072828498bb83e9dffaca389ad for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,767 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/9ae30df8bc6a4dc198df8a5bfb0a5075 for region: df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:10,768 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region df08d89907b6ac6a28b72ec117993ac2 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-05 15:54:10,812 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-05-05 15:54:10,813 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-05-05 15:54:10,815 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1683302050815"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1683302050815"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1683302050815"}]},"ts":"1683302050815"} 2023-05-05 15:54:10,815 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683302050815"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302050815"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302050815"}]},"ts":"1683302050815"} 2023-05-05 15:54:10,815 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683302050815"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302050815"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302050815"}]},"ts":"1683302050815"} 2023-05-05 15:54:10,818 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6cb2bd681dce908bd0c281b2c71d45e8, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2d9826f21eb94a559f0d5b68d98cb98d, ASSIGN}] 2023-05-05 15:54:10,819 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2d9826f21eb94a559f0d5b68d98cb98d, ASSIGN 2023-05-05 15:54:10,820 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2d9826f21eb94a559f0d5b68d98cb98d, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40991,1683302023016; forceNewPlan=false, retain=false 2023-05-05 15:54:10,820 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6cb2bd681dce908bd0c281b2c71d45e8, ASSIGN 2023-05-05 15:54:10,821 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6cb2bd681dce908bd0c281b2c71d45e8, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40991,1683302023016; forceNewPlan=false, retain=false 2023-05-05 15:54:10,970 INFO [jenkins-hbase4:44213] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-05 15:54:10,972 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=2d9826f21eb94a559f0d5b68d98cb98d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,972 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683302050972"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302050972"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302050972"}]},"ts":"1683302050972"} 2023-05-05 15:54:10,973 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; OpenRegionProcedure 2d9826f21eb94a559f0d5b68d98cb98d, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:54:10,976 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=6cb2bd681dce908bd0c281b2c71d45e8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:10,976 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683302050976"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302050976"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302050976"}]},"ts":"1683302050976"} 2023-05-05 15:54:10,977 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=75, state=RUNNABLE; OpenRegionProcedure 6cb2bd681dce908bd0c281b2c71d45e8, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:54:11,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d. 2023-05-05 15:54:11,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2d9826f21eb94a559f0d5b68d98cb98d, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.', STARTKEY => '60', ENDKEY => ''} 2023-05-05 15:54:11,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:54:11,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:54:11,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:54:11,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:54:11,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:54:11,130 INFO [StoreOpener-2d9826f21eb94a559f0d5b68d98cb98d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:54:11,131 DEBUG [StoreOpener-2d9826f21eb94a559f0d5b68d98cb98d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1 2023-05-05 15:54:11,132 DEBUG [StoreOpener-2d9826f21eb94a559f0d5b68d98cb98d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1 2023-05-05 15:54:11,132 INFO [StoreOpener-2d9826f21eb94a559f0d5b68d98cb98d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2d9826f21eb94a559f0d5b68d98cb98d columnFamilyName cf1 2023-05-05 15:54:11,142 DEBUG [StoreOpener-2d9826f21eb94a559f0d5b68d98cb98d-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/12a21e072828498bb83e9dffaca389ad.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/12a21e072828498bb83e9dffaca389ad-top 2023-05-05 15:54:11,146 DEBUG [StoreOpener-2d9826f21eb94a559f0d5b68d98cb98d-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/434afe9213bb430e9cb8198b4f2222f3.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/434afe9213bb430e9cb8198b4f2222f3-top 2023-05-05 15:54:11,149 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5d5c454854a3429ea5e9adedb86c8276.df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:11,149 DEBUG [StoreOpener-2d9826f21eb94a559f0d5b68d98cb98d-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/5d5c454854a3429ea5e9adedb86c8276.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/5d5c454854a3429ea5e9adedb86c8276-top 2023-05-05 15:54:11,153 DEBUG [StoreOpener-2d9826f21eb94a559f0d5b68d98cb98d-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/8593999d3ff54150bdb2dd76b2f46d3f.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/8593999d3ff54150bdb2dd76b2f46d3f-top 2023-05-05 15:54:11,157 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5e04725445fd4322809d1a9d85503ec1 2023-05-05 15:54:11,157 DEBUG [StoreOpener-2d9826f21eb94a559f0d5b68d98cb98d-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-5e04725445fd4322809d1a9d85503ec1 2023-05-05 15:54:11,159 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a0e50c325aef45bd913cb3678df787a7 2023-05-05 15:54:11,159 DEBUG [StoreOpener-2d9826f21eb94a559f0d5b68d98cb98d-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-a0e50c325aef45bd913cb3678df787a7 2023-05-05 15:54:11,162 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ab3a9bf885de4438ad1f504ac52c62c2 2023-05-05 15:54:11,162 DEBUG [StoreOpener-2d9826f21eb94a559f0d5b68d98cb98d-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-ab3a9bf885de4438ad1f504ac52c62c2 2023-05-05 15:54:11,162 INFO [StoreOpener-2d9826f21eb94a559f0d5b68d98cb98d-1] regionserver.HStore(310): Store=2d9826f21eb94a559f0d5b68d98cb98d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:54:11,162 INFO [StoreOpener-2d9826f21eb94a559f0d5b68d98cb98d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:54:11,163 DEBUG [StoreOpener-2d9826f21eb94a559f0d5b68d98cb98d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2 2023-05-05 15:54:11,163 DEBUG [StoreOpener-2d9826f21eb94a559f0d5b68d98cb98d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2 2023-05-05 15:54:11,163 INFO [StoreOpener-2d9826f21eb94a559f0d5b68d98cb98d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2d9826f21eb94a559f0d5b68d98cb98d columnFamilyName cf2 2023-05-05 15:54:11,173 DEBUG [StoreOpener-2d9826f21eb94a559f0d5b68d98cb98d-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/119b6673ef83418c87eba55400a3fa50.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/119b6673ef83418c87eba55400a3fa50-top 2023-05-05 15:54:11,176 DEBUG [StoreOpener-2d9826f21eb94a559f0d5b68d98cb98d-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/3d64d04b356c462c80aeb3d7341d16b5.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/3d64d04b356c462c80aeb3d7341d16b5-top 2023-05-05 15:54:11,180 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9ae30df8bc6a4dc198df8a5bfb0a5075.df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:11,181 DEBUG [StoreOpener-2d9826f21eb94a559f0d5b68d98cb98d-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/9ae30df8bc6a4dc198df8a5bfb0a5075.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/9ae30df8bc6a4dc198df8a5bfb0a5075-top 2023-05-05 15:54:11,184 DEBUG [StoreOpener-2d9826f21eb94a559f0d5b68d98cb98d-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/dd4d733d767c4c68bdcb726c005625f4.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/dd4d733d767c4c68bdcb726c005625f4-top 2023-05-05 15:54:11,187 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c31b04df32c4455aab59ebd1cada05ae 2023-05-05 15:54:11,187 DEBUG [StoreOpener-2d9826f21eb94a559f0d5b68d98cb98d-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-c31b04df32c4455aab59ebd1cada05ae 2023-05-05 15:54:11,190 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e7bd85555d934280bda43b21281bb981 2023-05-05 15:54:11,190 DEBUG [StoreOpener-2d9826f21eb94a559f0d5b68d98cb98d-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-e7bd85555d934280bda43b21281bb981 2023-05-05 15:54:11,194 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f1c90388d6a64950a4d581e92c36d16a 2023-05-05 15:54:11,194 DEBUG [StoreOpener-2d9826f21eb94a559f0d5b68d98cb98d-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-f1c90388d6a64950a4d581e92c36d16a 2023-05-05 15:54:11,194 INFO [StoreOpener-2d9826f21eb94a559f0d5b68d98cb98d-1] regionserver.HStore(310): Store=2d9826f21eb94a559f0d5b68d98cb98d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:54:11,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:54:11,196 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:54:11,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-05 15:54:11,199 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:54:11,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2d9826f21eb94a559f0d5b68d98cb98d; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74286879, jitterRate=0.10696075856685638}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-05 15:54:11,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2d9826f21eb94a559f0d5b68d98cb98d: 2023-05-05 15:54:11,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d., pid=77, masterSystemTime=1683302051125 2023-05-05 15:54:11,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:54:11,202 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-05 15:54:11,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:54:11,202 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-05 15:54:11,202 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d. 2023-05-05 15:54:11,202 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d. 2023-05-05 15:54:11,202 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1912): 2d9826f21eb94a559f0d5b68d98cb98d/cf1 is initiating minor compaction (all files) 2023-05-05 15:54:11,202 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1912): 2d9826f21eb94a559f0d5b68d98cb98d/cf2 is initiating minor compaction (all files) 2023-05-05 15:54:11,202 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2d9826f21eb94a559f0d5b68d98cb98d/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d. 2023-05-05 15:54:11,202 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 2d9826f21eb94a559f0d5b68d98cb98d/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d. 2023-05-05 15:54:11,202 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/12a21e072828498bb83e9dffaca389ad.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/12a21e072828498bb83e9dffaca389ad-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/434afe9213bb430e9cb8198b4f2222f3.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/434afe9213bb430e9cb8198b4f2222f3-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/8593999d3ff54150bdb2dd76b2f46d3f.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/8593999d3ff54150bdb2dd76b2f46d3f-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/5d5c454854a3429ea5e9adedb86c8276.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/5d5c454854a3429ea5e9adedb86c8276-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-a0e50c325aef45bd913cb3678df787a7, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-5e04725445fd4322809d1a9d85503ec1, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-ab3a9bf885de4438ad1f504ac52c62c2] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/.tmp, totalSize=34.8 K 2023-05-05 15:54:11,202 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/dd4d733d767c4c68bdcb726c005625f4.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/dd4d733d767c4c68bdcb726c005625f4-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/119b6673ef83418c87eba55400a3fa50.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/119b6673ef83418c87eba55400a3fa50-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/3d64d04b356c462c80aeb3d7341d16b5.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/3d64d04b356c462c80aeb3d7341d16b5-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/9ae30df8bc6a4dc198df8a5bfb0a5075.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/9ae30df8bc6a4dc198df8a5bfb0a5075-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-c31b04df32c4455aab59ebd1cada05ae, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-f1c90388d6a64950a4d581e92c36d16a, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-e7bd85555d934280bda43b21281bb981] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/.tmp, totalSize=34.8 K 2023-05-05 15:54:11,203 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting dd4d733d767c4c68bdcb726c005625f4.df08d89907b6ac6a28b72ec117993ac2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683302048586 2023-05-05 15:54:11,203 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 12a21e072828498bb83e9dffaca389ad.df08d89907b6ac6a28b72ec117993ac2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683302048586 2023-05-05 15:54:11,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d. 2023-05-05 15:54:11,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d. 2023-05-05 15:54:11,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8. 2023-05-05 15:54:11,204 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6cb2bd681dce908bd0c281b2c71d45e8, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.', STARTKEY => '', ENDKEY => '60'} 2023-05-05 15:54:11,204 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 434afe9213bb430e9cb8198b4f2222f3.df08d89907b6ac6a28b72ec117993ac2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683302048799 2023-05-05 15:54:11,204 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 119b6673ef83418c87eba55400a3fa50.df08d89907b6ac6a28b72ec117993ac2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683302048799 2023-05-05 15:54:11,204 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 8593999d3ff54150bdb2dd76b2f46d3f.df08d89907b6ac6a28b72ec117993ac2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683302048983 2023-05-05 15:54:11,204 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 3d64d04b356c462c80aeb3d7341d16b5.df08d89907b6ac6a28b72ec117993ac2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683302048983 2023-05-05 15:54:11,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:54:11,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 6cb2bd681dce908bd0c281b2c71d45e8 2023-05-05 15:54:11,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:54:11,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6cb2bd681dce908bd0c281b2c71d45e8 2023-05-05 15:54:11,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6cb2bd681dce908bd0c281b2c71d45e8 2023-05-05 15:54:11,205 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=2d9826f21eb94a559f0d5b68d98cb98d, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:11,205 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 5d5c454854a3429ea5e9adedb86c8276.df08d89907b6ac6a28b72ec117993ac2, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1683302049746 2023-05-05 15:54:11,205 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683302051205"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302051205"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302051205"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302051205"}]},"ts":"1683302051205"} 2023-05-05 15:54:11,208 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-05-05 15:54:11,208 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; OpenRegionProcedure 2d9826f21eb94a559f0d5b68d98cb98d, server=jenkins-hbase4.apache.org,40991,1683302023016 in 234 msec 2023-05-05 15:54:11,209 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-a0e50c325aef45bd913cb3678df787a7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-05-05 15:54:11,209 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 9ae30df8bc6a4dc198df8a5bfb0a5075.df08d89907b6ac6a28b72ec117993ac2, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1683302049746 2023-05-05 15:54:11,209 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-5e04725445fd4322809d1a9d85503ec1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-05-05 15:54:11,209 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-c31b04df32c4455aab59ebd1cada05ae, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-05-05 15:54:11,210 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2d9826f21eb94a559f0d5b68d98cb98d, ASSIGN in 390 msec 2023-05-05 15:54:11,210 INFO [StoreOpener-6cb2bd681dce908bd0c281b2c71d45e8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6cb2bd681dce908bd0c281b2c71d45e8 2023-05-05 15:54:11,210 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-ab3a9bf885de4438ad1f504ac52c62c2, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-05-05 15:54:11,211 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-f1c90388d6a64950a4d581e92c36d16a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-05-05 15:54:11,211 DEBUG [StoreOpener-6cb2bd681dce908bd0c281b2c71d45e8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1 2023-05-05 15:54:11,211 DEBUG [StoreOpener-6cb2bd681dce908bd0c281b2c71d45e8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1 2023-05-05 15:54:11,211 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-e7bd85555d934280bda43b21281bb981, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-05-05 15:54:11,211 INFO [StoreOpener-6cb2bd681dce908bd0c281b2c71d45e8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6cb2bd681dce908bd0c281b2c71d45e8 columnFamilyName cf1 2023-05-05 15:54:11,225 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2d9826f21eb94a559f0d5b68d98cb98d#cf1#compaction#72 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:54:11,228 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] throttle.PressureAwareThroughputController(145): 2d9826f21eb94a559f0d5b68d98cb98d#cf2#compaction#73 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:54:11,233 DEBUG [StoreOpener-6cb2bd681dce908bd0c281b2c71d45e8-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/12a21e072828498bb83e9dffaca389ad.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/12a21e072828498bb83e9dffaca389ad-bottom 2023-05-05 15:54:11,236 DEBUG [StoreOpener-6cb2bd681dce908bd0c281b2c71d45e8-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/434afe9213bb430e9cb8198b4f2222f3.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/434afe9213bb430e9cb8198b4f2222f3-bottom 2023-05-05 15:54:11,239 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5d5c454854a3429ea5e9adedb86c8276.df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:11,239 DEBUG [StoreOpener-6cb2bd681dce908bd0c281b2c71d45e8-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/5d5c454854a3429ea5e9adedb86c8276.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/5d5c454854a3429ea5e9adedb86c8276-bottom 2023-05-05 15:54:11,242 DEBUG [StoreOpener-6cb2bd681dce908bd0c281b2c71d45e8-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/8593999d3ff54150bdb2dd76b2f46d3f.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/8593999d3ff54150bdb2dd76b2f46d3f-bottom 2023-05-05 15:54:11,244 DEBUG [StoreOpener-6cb2bd681dce908bd0c281b2c71d45e8-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-de1a5ea53f2a42578b0dcd4c71037963 2023-05-05 15:54:11,250 DEBUG [StoreOpener-6cb2bd681dce908bd0c281b2c71d45e8-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-e68abacff98449d58bd61ca4ffe85094 2023-05-05 15:54:11,251 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-05 15:54:11,253 DEBUG [StoreOpener-6cb2bd681dce908bd0c281b2c71d45e8-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-f87f9f855cfc4676bdef6c17cad64e1c 2023-05-05 15:54:11,259 INFO [StoreOpener-6cb2bd681dce908bd0c281b2c71d45e8-1] regionserver.HStore(310): Store=6cb2bd681dce908bd0c281b2c71d45e8/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:54:11,259 INFO [StoreOpener-6cb2bd681dce908bd0c281b2c71d45e8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6cb2bd681dce908bd0c281b2c71d45e8 2023-05-05 15:54:11,261 DEBUG [StoreOpener-6cb2bd681dce908bd0c281b2c71d45e8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2 2023-05-05 15:54:11,261 DEBUG [StoreOpener-6cb2bd681dce908bd0c281b2c71d45e8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2 2023-05-05 15:54:11,262 INFO [StoreOpener-6cb2bd681dce908bd0c281b2c71d45e8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6cb2bd681dce908bd0c281b2c71d45e8 columnFamilyName cf2 2023-05-05 15:54:11,268 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for e0bc3e7deb3b4f14a2dfa6a696f9b4de: NONE, but ROW specified in column family configuration 2023-05-05 15:54:11,270 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for f43d91500a9940a89c3e72883528ac26: NONE, but ROW specified in column family configuration 2023-05-05 15:54:11,270 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/.tmp/cf1/e0bc3e7deb3b4f14a2dfa6a696f9b4de as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/e0bc3e7deb3b4f14a2dfa6a696f9b4de 2023-05-05 15:54:11,273 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/.tmp/cf2/f43d91500a9940a89c3e72883528ac26 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/f43d91500a9940a89c3e72883528ac26 2023-05-05 15:54:11,277 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for e0bc3e7deb3b4f14a2dfa6a696f9b4de: NONE, but ROW specified in column family configuration 2023-05-05 15:54:11,278 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 2d9826f21eb94a559f0d5b68d98cb98d/cf1 of 2d9826f21eb94a559f0d5b68d98cb98d into e0bc3e7deb3b4f14a2dfa6a696f9b4de(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:54:11,278 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for f43d91500a9940a89c3e72883528ac26: NONE, but ROW specified in column family configuration 2023-05-05 15:54:11,278 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2d9826f21eb94a559f0d5b68d98cb98d: 2023-05-05 15:54:11,279 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d., storeName=2d9826f21eb94a559f0d5b68d98cb98d/cf1, priority=9, startTime=1683302051201; duration=0sec 2023-05-05 15:54:11,279 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:54:11,279 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 2d9826f21eb94a559f0d5b68d98cb98d/cf2 of 2d9826f21eb94a559f0d5b68d98cb98d into f43d91500a9940a89c3e72883528ac26(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:54:11,279 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2d9826f21eb94a559f0d5b68d98cb98d: 2023-05-05 15:54:11,279 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d., storeName=2d9826f21eb94a559f0d5b68d98cb98d/cf2, priority=9, startTime=1683302051202; duration=0sec 2023-05-05 15:54:11,280 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:54:11,281 DEBUG [StoreOpener-6cb2bd681dce908bd0c281b2c71d45e8-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/119b6673ef83418c87eba55400a3fa50.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/119b6673ef83418c87eba55400a3fa50-bottom 2023-05-05 15:54:11,284 DEBUG [StoreOpener-6cb2bd681dce908bd0c281b2c71d45e8-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/3d64d04b356c462c80aeb3d7341d16b5.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/3d64d04b356c462c80aeb3d7341d16b5-bottom 2023-05-05 15:54:11,288 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9ae30df8bc6a4dc198df8a5bfb0a5075.df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:11,289 DEBUG [StoreOpener-6cb2bd681dce908bd0c281b2c71d45e8-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/9ae30df8bc6a4dc198df8a5bfb0a5075.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/9ae30df8bc6a4dc198df8a5bfb0a5075-bottom 2023-05-05 15:54:11,297 DEBUG [StoreOpener-6cb2bd681dce908bd0c281b2c71d45e8-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/dd4d733d767c4c68bdcb726c005625f4.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/dd4d733d767c4c68bdcb726c005625f4-bottom 2023-05-05 15:54:11,302 DEBUG [StoreOpener-6cb2bd681dce908bd0c281b2c71d45e8-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-5dbf1121ee934ff395d553d9c0733a84 2023-05-05 15:54:11,305 DEBUG [StoreOpener-6cb2bd681dce908bd0c281b2c71d45e8-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-5dfe0faf8f924ef29b2ab0594e62a7bc 2023-05-05 15:54:11,309 DEBUG [StoreOpener-6cb2bd681dce908bd0c281b2c71d45e8-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-f775112a6fc94a92b672ce0af66e793b 2023-05-05 15:54:11,309 INFO [StoreOpener-6cb2bd681dce908bd0c281b2c71d45e8-1] regionserver.HStore(310): Store=6cb2bd681dce908bd0c281b2c71d45e8/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:54:11,310 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8 2023-05-05 15:54:11,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8 2023-05-05 15:54:11,312 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-05 15:54:11,313 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6cb2bd681dce908bd0c281b2c71d45e8 2023-05-05 15:54:11,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6cb2bd681dce908bd0c281b2c71d45e8; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70290183, jitterRate=0.04740534722805023}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-05 15:54:11,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6cb2bd681dce908bd0c281b2c71d45e8: 2023-05-05 15:54:11,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8., pid=78, masterSystemTime=1683302051125 2023-05-05 15:54:11,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:54:11,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-05 15:54:11,316 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-05 15:54:11,316 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-05 15:54:11,316 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8. 2023-05-05 15:54:11,316 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8. 2023-05-05 15:54:11,316 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1912): 6cb2bd681dce908bd0c281b2c71d45e8/cf1 is initiating minor compaction (all files) 2023-05-05 15:54:11,316 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1912): 6cb2bd681dce908bd0c281b2c71d45e8/cf2 is initiating minor compaction (all files) 2023-05-05 15:54:11,317 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 6cb2bd681dce908bd0c281b2c71d45e8/cf1 in testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8. 2023-05-05 15:54:11,317 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 6cb2bd681dce908bd0c281b2c71d45e8/cf2 in testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8. 2023-05-05 15:54:11,317 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/12a21e072828498bb83e9dffaca389ad.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/12a21e072828498bb83e9dffaca389ad-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/434afe9213bb430e9cb8198b4f2222f3.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/434afe9213bb430e9cb8198b4f2222f3-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/8593999d3ff54150bdb2dd76b2f46d3f.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/8593999d3ff54150bdb2dd76b2f46d3f-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-f87f9f855cfc4676bdef6c17cad64e1c, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-de1a5ea53f2a42578b0dcd4c71037963, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-e68abacff98449d58bd61ca4ffe85094, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/5d5c454854a3429ea5e9adedb86c8276.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/5d5c454854a3429ea5e9adedb86c8276-bottom] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/.tmp, totalSize=35.0 K 2023-05-05 15:54:11,317 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/dd4d733d767c4c68bdcb726c005625f4.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/dd4d733d767c4c68bdcb726c005625f4-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/119b6673ef83418c87eba55400a3fa50.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/119b6673ef83418c87eba55400a3fa50-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/3d64d04b356c462c80aeb3d7341d16b5.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/3d64d04b356c462c80aeb3d7341d16b5-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-5dbf1121ee934ff395d553d9c0733a84, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-5dfe0faf8f924ef29b2ab0594e62a7bc, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-f775112a6fc94a92b672ce0af66e793b, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/9ae30df8bc6a4dc198df8a5bfb0a5075.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/9ae30df8bc6a4dc198df8a5bfb0a5075-bottom] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/.tmp, totalSize=35.0 K 2023-05-05 15:54:11,318 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 12a21e072828498bb83e9dffaca389ad.df08d89907b6ac6a28b72ec117993ac2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683302048586 2023-05-05 15:54:11,318 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8. 2023-05-05 15:54:11,318 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting dd4d733d767c4c68bdcb726c005625f4.df08d89907b6ac6a28b72ec117993ac2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683302048586 2023-05-05 15:54:11,318 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 434afe9213bb430e9cb8198b4f2222f3.df08d89907b6ac6a28b72ec117993ac2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683302048799 2023-05-05 15:54:11,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8. 2023-05-05 15:54:11,319 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 119b6673ef83418c87eba55400a3fa50.df08d89907b6ac6a28b72ec117993ac2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683302048799 2023-05-05 15:54:11,319 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=6cb2bd681dce908bd0c281b2c71d45e8, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:11,319 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 3d64d04b356c462c80aeb3d7341d16b5.df08d89907b6ac6a28b72ec117993ac2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683302048983 2023-05-05 15:54:11,319 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 8593999d3ff54150bdb2dd76b2f46d3f.df08d89907b6ac6a28b72ec117993ac2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683302048983 2023-05-05 15:54:11,319 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683302051319"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302051319"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302051319"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302051319"}]},"ts":"1683302051319"} 2023-05-05 15:54:11,319 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-5dbf1121ee934ff395d553d9c0733a84, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683302049177 2023-05-05 15:54:11,319 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-f87f9f855cfc4676bdef6c17cad64e1c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683302049177 2023-05-05 15:54:11,320 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-5dfe0faf8f924ef29b2ab0594e62a7bc, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683302049368 2023-05-05 15:54:11,320 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-de1a5ea53f2a42578b0dcd4c71037963, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683302049368 2023-05-05 15:54:11,320 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-f775112a6fc94a92b672ce0af66e793b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683302049553 2023-05-05 15:54:11,320 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-e68abacff98449d58bd61ca4ffe85094, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683302049553 2023-05-05 15:54:11,321 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 9ae30df8bc6a4dc198df8a5bfb0a5075.df08d89907b6ac6a28b72ec117993ac2, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683302049746 2023-05-05 15:54:11,321 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 5d5c454854a3429ea5e9adedb86c8276.df08d89907b6ac6a28b72ec117993ac2, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683302049746 2023-05-05 15:54:11,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=75 2023-05-05 15:54:11,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=75, state=SUCCESS; OpenRegionProcedure 6cb2bd681dce908bd0c281b2c71d45e8, server=jenkins-hbase4.apache.org,40991,1683302023016 in 343 msec 2023-05-05 15:54:11,324 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=75, resume processing ppid=72 2023-05-05 15:54:11,324 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6cb2bd681dce908bd0c281b2c71d45e8, ASSIGN in 505 msec 2023-05-05 15:54:11,326 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=df08d89907b6ac6a28b72ec117993ac2, daughterA=6cb2bd681dce908bd0c281b2c71d45e8, daughterB=2d9826f21eb94a559f0d5b68d98cb98d in 1.0410 sec 2023-05-05 15:54:11,336 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6cb2bd681dce908bd0c281b2c71d45e8#cf1#compaction#74 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:54:11,340 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] throttle.PressureAwareThroughputController(145): 6cb2bd681dce908bd0c281b2c71d45e8#cf2#compaction#75 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:54:11,359 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/.tmp/cf2/80af74b744af49e8ac01886dae95befc as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/80af74b744af49e8ac01886dae95befc 2023-05-05 15:54:11,359 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/.tmp/cf1/33d33863af674a8ab0f71a9fdd88eb19 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/33d33863af674a8ab0f71a9fdd88eb19 2023-05-05 15:54:11,364 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6cb2bd681dce908bd0c281b2c71d45e8/cf2 of 6cb2bd681dce908bd0c281b2c71d45e8 into 80af74b744af49e8ac01886dae95befc(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:54:11,364 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6cb2bd681dce908bd0c281b2c71d45e8: 2023-05-05 15:54:11,364 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8., storeName=6cb2bd681dce908bd0c281b2c71d45e8/cf2, priority=9, startTime=1683302051316; duration=0sec 2023-05-05 15:54:11,364 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6cb2bd681dce908bd0c281b2c71d45e8/cf1 of 6cb2bd681dce908bd0c281b2c71d45e8 into 33d33863af674a8ab0f71a9fdd88eb19(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:54:11,364 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:54:11,364 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6cb2bd681dce908bd0c281b2c71d45e8: 2023-05-05 15:54:11,364 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8., storeName=6cb2bd681dce908bd0c281b2c71d45e8/cf1, priority=9, startTime=1683302051315; duration=0sec 2023-05-05 15:54:11,364 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:54:11,545 DEBUG [Listener at localhost/35117] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-05-05 15:54:11,548 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49936, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-05-05 15:54:11,548 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40991] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8. 2023-05-05 15:54:11,549 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40991] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-05-05 15:54:11,549 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40991] regionserver.HStore(1912): 6cb2bd681dce908bd0c281b2c71d45e8/cf1 is initiating major compaction (all files) 2023-05-05 15:54:11,549 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40991] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7394e8d6; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:54:11,549 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 6cb2bd681dce908bd0c281b2c71d45e8/cf1 in testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8. 2023-05-05 15:54:11,549 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40991] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-05-05 15:54:11,549 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/33d33863af674a8ab0f71a9fdd88eb19] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/.tmp, totalSize=6.8 K 2023-05-05 15:54:11,549 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40991] regionserver.HStore(1912): 6cb2bd681dce908bd0c281b2c71d45e8/cf2 is initiating major compaction (all files) 2023-05-05 15:54:11,549 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40991] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1380b62d; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-05 15:54:11,550 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 6cb2bd681dce908bd0c281b2c71d45e8/cf2 in testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8. 2023-05-05 15:54:11,550 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/80af74b744af49e8ac01886dae95befc] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/.tmp, totalSize=6.8 K 2023-05-05 15:54:11,550 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 33d33863af674a8ab0f71a9fdd88eb19, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683302048586 2023-05-05 15:54:11,550 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 80af74b744af49e8ac01886dae95befc, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683302048586 2023-05-05 15:54:11,551 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40991] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d. 2023-05-05 15:54:11,551 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40991] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-05-05 15:54:11,552 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40991] regionserver.HStore(1912): 2d9826f21eb94a559f0d5b68d98cb98d/cf1 is initiating major compaction (all files) 2023-05-05 15:54:11,552 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40991] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@38a9ee9; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-05 15:54:11,552 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40991] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-05-05 15:54:11,552 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40991] regionserver.HStore(1912): 2d9826f21eb94a559f0d5b68d98cb98d/cf2 is initiating major compaction (all files) 2023-05-05 15:54:11,552 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40991] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3aa7f39a; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-05-05 15:54:11,553 INFO [Listener at localhost/35117] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-05-05 15:54:11,555 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6cb2bd681dce908bd0c281b2c71d45e8#cf1#compaction#76 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:54:11,555 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] throttle.PressureAwareThroughputController(145): 6cb2bd681dce908bd0c281b2c71d45e8#cf2#compaction#77 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:54:11,570 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40991] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8. is not mergeable because it has references 2023-05-05 15:54:11,571 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40991] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d. is not mergeable because it has references 2023-05-05 15:54:11,577 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/.tmp/cf1/5a7fa4dc6aec484db8fa84fb0491c8e8 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/5a7fa4dc6aec484db8fa84fb0491c8e8 2023-05-05 15:54:11,578 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/.tmp/cf2/42013ccea87d4b4faa5251137e4fe3d0 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/42013ccea87d4b4faa5251137e4fe3d0 2023-05-05 15:54:11,581 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 6cb2bd681dce908bd0c281b2c71d45e8/cf1 of 6cb2bd681dce908bd0c281b2c71d45e8 into 5a7fa4dc6aec484db8fa84fb0491c8e8(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:54:11,582 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6cb2bd681dce908bd0c281b2c71d45e8: 2023-05-05 15:54:11,583 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1683302051549; duration=0sec 2023-05-05 15:54:11,583 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 6cb2bd681dce908bd0c281b2c71d45e8/cf2 of 6cb2bd681dce908bd0c281b2c71d45e8 into 42013ccea87d4b4faa5251137e4fe3d0(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:54:11,583 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-05-05 15:54:11,583 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6cb2bd681dce908bd0c281b2c71d45e8: 2023-05-05 15:54:11,583 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2d9826f21eb94a559f0d5b68d98cb98d/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d. 2023-05-05 15:54:11,583 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1683302051549; duration=0sec 2023-05-05 15:54:11,583 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/e0bc3e7deb3b4f14a2dfa6a696f9b4de] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/.tmp, totalSize=5.0 K 2023-05-05 15:54:11,583 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-05 15:54:11,583 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 2d9826f21eb94a559f0d5b68d98cb98d/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d. 2023-05-05 15:54:11,583 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/f43d91500a9940a89c3e72883528ac26] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/.tmp, totalSize=5.0 K 2023-05-05 15:54:11,583 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting e0bc3e7deb3b4f14a2dfa6a696f9b4de, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-05-05 15:54:11,584 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting f43d91500a9940a89c3e72883528ac26, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-05-05 15:54:11,586 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2d9826f21eb94a559f0d5b68d98cb98d#cf1#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:54:11,586 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] throttle.PressureAwareThroughputController(145): 2d9826f21eb94a559f0d5b68d98cb98d#cf2#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:54:11,605 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for c94537c5b50f40ceb98e230ab99b029d: NONE, but ROW specified in column family configuration 2023-05-05 15:54:11,605 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 7c95b060f02640caa9fa32d1075a9174: NONE, but ROW specified in column family configuration 2023-05-05 15:54:11,605 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/.tmp/cf1/c94537c5b50f40ceb98e230ab99b029d as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/c94537c5b50f40ceb98e230ab99b029d 2023-05-05 15:54:11,606 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/.tmp/cf2/7c95b060f02640caa9fa32d1075a9174 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/7c95b060f02640caa9fa32d1075a9174 2023-05-05 15:54:11,609 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for c94537c5b50f40ceb98e230ab99b029d: NONE, but ROW specified in column family configuration 2023-05-05 15:54:11,609 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 7c95b060f02640caa9fa32d1075a9174: NONE, but ROW specified in column family configuration 2023-05-05 15:54:11,610 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 2d9826f21eb94a559f0d5b68d98cb98d/cf1 of 2d9826f21eb94a559f0d5b68d98cb98d into c94537c5b50f40ceb98e230ab99b029d(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:54:11,610 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2d9826f21eb94a559f0d5b68d98cb98d: 2023-05-05 15:54:11,610 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1683302051551; duration=0sec 2023-05-05 15:54:11,610 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:54:11,611 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 2d9826f21eb94a559f0d5b68d98cb98d/cf2 of 2d9826f21eb94a559f0d5b68d98cb98d into 7c95b060f02640caa9fa32d1075a9174(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:54:11,611 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2d9826f21eb94a559f0d5b68d98cb98d: 2023-05-05 15:54:11,611 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1683302051552; duration=0sec 2023-05-05 15:54:11,611 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:54:11,679 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40991] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8. is not mergeable because it has references 2023-05-05 15:54:11,680 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40991] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d. is not mergeable because it has references 2023-05-05 15:54:11,686 DEBUG [Listener at localhost/35117] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-05-05 15:54:11,688 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-05 15:54:11,689 INFO [Listener at localhost/35117] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:11,690 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:11,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:11,693 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-05-05 15:54:11,693 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302051693"}]},"ts":"1683302051693"} 2023-05-05 15:54:11,695 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-05-05 15:54:11,697 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-05-05 15:54:11,698 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6cb2bd681dce908bd0c281b2c71d45e8, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2d9826f21eb94a559f0d5b68d98cb98d, UNASSIGN}] 2023-05-05 15:54:11,698 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2d9826f21eb94a559f0d5b68d98cb98d, UNASSIGN 2023-05-05 15:54:11,699 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=2d9826f21eb94a559f0d5b68d98cb98d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:11,699 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683302051699"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302051699"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302051699"}]},"ts":"1683302051699"} 2023-05-05 15:54:11,700 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure 2d9826f21eb94a559f0d5b68d98cb98d, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:54:11,703 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6cb2bd681dce908bd0c281b2c71d45e8, UNASSIGN 2023-05-05 15:54:11,703 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=6cb2bd681dce908bd0c281b2c71d45e8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:11,704 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683302051703"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302051703"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302051703"}]},"ts":"1683302051703"} 2023-05-05 15:54:11,705 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure 6cb2bd681dce908bd0c281b2c71d45e8, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:54:11,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-05-05 15:54:11,852 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 6cb2bd681dce908bd0c281b2c71d45e8 2023-05-05 15:54:11,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6cb2bd681dce908bd0c281b2c71d45e8, disabling compactions & flushes 2023-05-05 15:54:11,852 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8. 2023-05-05 15:54:11,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8. 2023-05-05 15:54:11,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8. after waiting 0 ms 2023-05-05 15:54:11,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8. 2023-05-05 15:54:11,853 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/12a21e072828498bb83e9dffaca389ad.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/12a21e072828498bb83e9dffaca389ad-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/434afe9213bb430e9cb8198b4f2222f3.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/434afe9213bb430e9cb8198b4f2222f3-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/8593999d3ff54150bdb2dd76b2f46d3f.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/8593999d3ff54150bdb2dd76b2f46d3f-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-f87f9f855cfc4676bdef6c17cad64e1c, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-de1a5ea53f2a42578b0dcd4c71037963, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-e68abacff98449d58bd61ca4ffe85094, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/33d33863af674a8ab0f71a9fdd88eb19, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/5d5c454854a3429ea5e9adedb86c8276.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/5d5c454854a3429ea5e9adedb86c8276-bottom] to archive 2023-05-05 15:54:11,854 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-05 15:54:11,856 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/12a21e072828498bb83e9dffaca389ad.df08d89907b6ac6a28b72ec117993ac2 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/12a21e072828498bb83e9dffaca389ad.df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:11,857 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/434afe9213bb430e9cb8198b4f2222f3.df08d89907b6ac6a28b72ec117993ac2 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/434afe9213bb430e9cb8198b4f2222f3.df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:11,858 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/8593999d3ff54150bdb2dd76b2f46d3f.df08d89907b6ac6a28b72ec117993ac2 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/8593999d3ff54150bdb2dd76b2f46d3f.df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:11,859 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-f87f9f855cfc4676bdef6c17cad64e1c to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-f87f9f855cfc4676bdef6c17cad64e1c 2023-05-05 15:54:11,860 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-de1a5ea53f2a42578b0dcd4c71037963 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-de1a5ea53f2a42578b0dcd4c71037963 2023-05-05 15:54:11,862 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-e68abacff98449d58bd61ca4ffe85094 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-e68abacff98449d58bd61ca4ffe85094 2023-05-05 15:54:11,863 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/33d33863af674a8ab0f71a9fdd88eb19 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/33d33863af674a8ab0f71a9fdd88eb19 2023-05-05 15:54:11,864 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/5d5c454854a3429ea5e9adedb86c8276.df08d89907b6ac6a28b72ec117993ac2 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/5d5c454854a3429ea5e9adedb86c8276.df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:11,865 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/dd4d733d767c4c68bdcb726c005625f4.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/dd4d733d767c4c68bdcb726c005625f4-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/119b6673ef83418c87eba55400a3fa50.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/119b6673ef83418c87eba55400a3fa50-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/3d64d04b356c462c80aeb3d7341d16b5.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/3d64d04b356c462c80aeb3d7341d16b5-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-5dbf1121ee934ff395d553d9c0733a84, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-5dfe0faf8f924ef29b2ab0594e62a7bc, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-f775112a6fc94a92b672ce0af66e793b, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/80af74b744af49e8ac01886dae95befc, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/9ae30df8bc6a4dc198df8a5bfb0a5075.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/9ae30df8bc6a4dc198df8a5bfb0a5075-bottom] to archive 2023-05-05 15:54:11,866 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-05 15:54:11,867 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/dd4d733d767c4c68bdcb726c005625f4.df08d89907b6ac6a28b72ec117993ac2 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/dd4d733d767c4c68bdcb726c005625f4.df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:11,868 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/119b6673ef83418c87eba55400a3fa50.df08d89907b6ac6a28b72ec117993ac2 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/119b6673ef83418c87eba55400a3fa50.df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:11,869 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/3d64d04b356c462c80aeb3d7341d16b5.df08d89907b6ac6a28b72ec117993ac2 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/3d64d04b356c462c80aeb3d7341d16b5.df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:11,870 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-5dbf1121ee934ff395d553d9c0733a84 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-5dbf1121ee934ff395d553d9c0733a84 2023-05-05 15:54:11,871 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-5dfe0faf8f924ef29b2ab0594e62a7bc to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-5dfe0faf8f924ef29b2ab0594e62a7bc 2023-05-05 15:54:11,872 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-f775112a6fc94a92b672ce0af66e793b to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-f775112a6fc94a92b672ce0af66e793b 2023-05-05 15:54:11,873 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/80af74b744af49e8ac01886dae95befc to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/80af74b744af49e8ac01886dae95befc 2023-05-05 15:54:11,874 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/9ae30df8bc6a4dc198df8a5bfb0a5075.df08d89907b6ac6a28b72ec117993ac2 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/9ae30df8bc6a4dc198df8a5bfb0a5075.df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:11,878 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-05-05 15:54:11,878 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:54:11,880 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8. 2023-05-05 15:54:11,880 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6cb2bd681dce908bd0c281b2c71d45e8: 2023-05-05 15:54:11,881 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 6cb2bd681dce908bd0c281b2c71d45e8 2023-05-05 15:54:11,881 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:54:11,881 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2d9826f21eb94a559f0d5b68d98cb98d, disabling compactions & flushes 2023-05-05 15:54:11,881 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d. 2023-05-05 15:54:11,881 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d. 2023-05-05 15:54:11,881 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d. after waiting 0 ms 2023-05-05 15:54:11,881 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d. 2023-05-05 15:54:11,881 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=6cb2bd681dce908bd0c281b2c71d45e8, regionState=CLOSED 2023-05-05 15:54:11,881 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683302051881"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302051881"}]},"ts":"1683302051881"} 2023-05-05 15:54:11,882 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/12a21e072828498bb83e9dffaca389ad.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/12a21e072828498bb83e9dffaca389ad-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/434afe9213bb430e9cb8198b4f2222f3.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/434afe9213bb430e9cb8198b4f2222f3-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/8593999d3ff54150bdb2dd76b2f46d3f.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/8593999d3ff54150bdb2dd76b2f46d3f-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/5d5c454854a3429ea5e9adedb86c8276.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/5d5c454854a3429ea5e9adedb86c8276-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-a0e50c325aef45bd913cb3678df787a7, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-5e04725445fd4322809d1a9d85503ec1, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/e0bc3e7deb3b4f14a2dfa6a696f9b4de, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-ab3a9bf885de4438ad1f504ac52c62c2] to archive 2023-05-05 15:54:11,883 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-05 15:54:11,884 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-05-05 15:54:11,884 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure 6cb2bd681dce908bd0c281b2c71d45e8, server=jenkins-hbase4.apache.org,40991,1683302023016 in 178 msec 2023-05-05 15:54:11,884 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/12a21e072828498bb83e9dffaca389ad.df08d89907b6ac6a28b72ec117993ac2 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/12a21e072828498bb83e9dffaca389ad.df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:11,885 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6cb2bd681dce908bd0c281b2c71d45e8, UNASSIGN in 186 msec 2023-05-05 15:54:11,886 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/434afe9213bb430e9cb8198b4f2222f3.df08d89907b6ac6a28b72ec117993ac2 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/434afe9213bb430e9cb8198b4f2222f3.df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:11,887 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/8593999d3ff54150bdb2dd76b2f46d3f.df08d89907b6ac6a28b72ec117993ac2 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/8593999d3ff54150bdb2dd76b2f46d3f.df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:11,888 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/5d5c454854a3429ea5e9adedb86c8276.df08d89907b6ac6a28b72ec117993ac2 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/5d5c454854a3429ea5e9adedb86c8276.df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:11,889 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-a0e50c325aef45bd913cb3678df787a7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-a0e50c325aef45bd913cb3678df787a7 2023-05-05 15:54:11,890 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-5e04725445fd4322809d1a9d85503ec1 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-5e04725445fd4322809d1a9d85503ec1 2023-05-05 15:54:11,891 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/e0bc3e7deb3b4f14a2dfa6a696f9b4de to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/e0bc3e7deb3b4f14a2dfa6a696f9b4de 2023-05-05 15:54:11,892 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-ab3a9bf885de4438ad1f504ac52c62c2 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-ab3a9bf885de4438ad1f504ac52c62c2 2023-05-05 15:54:11,893 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/dd4d733d767c4c68bdcb726c005625f4.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/dd4d733d767c4c68bdcb726c005625f4-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/119b6673ef83418c87eba55400a3fa50.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/119b6673ef83418c87eba55400a3fa50-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/3d64d04b356c462c80aeb3d7341d16b5.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/3d64d04b356c462c80aeb3d7341d16b5-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/9ae30df8bc6a4dc198df8a5bfb0a5075.df08d89907b6ac6a28b72ec117993ac2->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/9ae30df8bc6a4dc198df8a5bfb0a5075-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-c31b04df32c4455aab59ebd1cada05ae, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-f1c90388d6a64950a4d581e92c36d16a, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/f43d91500a9940a89c3e72883528ac26, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-e7bd85555d934280bda43b21281bb981] to archive 2023-05-05 15:54:11,894 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-05 15:54:11,895 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/dd4d733d767c4c68bdcb726c005625f4.df08d89907b6ac6a28b72ec117993ac2 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/dd4d733d767c4c68bdcb726c005625f4.df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:11,896 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/119b6673ef83418c87eba55400a3fa50.df08d89907b6ac6a28b72ec117993ac2 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/119b6673ef83418c87eba55400a3fa50.df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:11,897 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/3d64d04b356c462c80aeb3d7341d16b5.df08d89907b6ac6a28b72ec117993ac2 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/3d64d04b356c462c80aeb3d7341d16b5.df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:11,898 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/9ae30df8bc6a4dc198df8a5bfb0a5075.df08d89907b6ac6a28b72ec117993ac2 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/9ae30df8bc6a4dc198df8a5bfb0a5075.df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:11,899 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-c31b04df32c4455aab59ebd1cada05ae to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-c31b04df32c4455aab59ebd1cada05ae 2023-05-05 15:54:11,900 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-f1c90388d6a64950a4d581e92c36d16a to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-f1c90388d6a64950a4d581e92c36d16a 2023-05-05 15:54:11,901 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/f43d91500a9940a89c3e72883528ac26 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/f43d91500a9940a89c3e72883528ac26 2023-05-05 15:54:11,903 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-e7bd85555d934280bda43b21281bb981 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/testSplitTableRegionDeletedRowsDaughter=df08d89907b6ac6a28b72ec117993ac2-e7bd85555d934280bda43b21281bb981 2023-05-05 15:54:11,907 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-05-05 15:54:11,907 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:54:11,909 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d. 2023-05-05 15:54:11,909 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2d9826f21eb94a559f0d5b68d98cb98d: 2023-05-05 15:54:11,910 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:54:11,910 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=2d9826f21eb94a559f0d5b68d98cb98d, regionState=CLOSED 2023-05-05 15:54:11,911 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683302051910"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302051910"}]},"ts":"1683302051910"} 2023-05-05 15:54:11,913 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-05-05 15:54:11,913 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure 2d9826f21eb94a559f0d5b68d98cb98d, server=jenkins-hbase4.apache.org,40991,1683302023016 in 212 msec 2023-05-05 15:54:11,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=79 2023-05-05 15:54:11,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2d9826f21eb94a559f0d5b68d98cb98d, UNASSIGN in 215 msec 2023-05-05 15:54:11,915 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302051915"}]},"ts":"1683302051915"} 2023-05-05 15:54:11,916 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-05-05 15:54:11,918 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-05-05 15:54:11,919 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 228 msec 2023-05-05 15:54:11,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-05-05 15:54:11,995 INFO [Listener at localhost/35117] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-05-05 15:54:11,996 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:11,997 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:11,999 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:11,999 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-05-05 15:54:11,999 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:12,004 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:12,004 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8 2023-05-05 15:54:12,004 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:54:12,006 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/recovered.edits] 2023-05-05 15:54:12,006 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/recovered.edits] 2023-05-05 15:54:12,006 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/recovered.edits] 2023-05-05 15:54:12,009 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/5a7fa4dc6aec484db8fa84fb0491c8e8 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf1/5a7fa4dc6aec484db8fa84fb0491c8e8 2023-05-05 15:54:12,009 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/12a21e072828498bb83e9dffaca389ad to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/12a21e072828498bb83e9dffaca389ad 2023-05-05 15:54:12,009 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/c94537c5b50f40ceb98e230ab99b029d to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf1/c94537c5b50f40ceb98e230ab99b029d 2023-05-05 15:54:12,011 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/434afe9213bb430e9cb8198b4f2222f3 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/434afe9213bb430e9cb8198b4f2222f3 2023-05-05 15:54:12,012 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/42013ccea87d4b4faa5251137e4fe3d0 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/cf2/42013ccea87d4b4faa5251137e4fe3d0 2023-05-05 15:54:12,012 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/7c95b060f02640caa9fa32d1075a9174 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/cf2/7c95b060f02640caa9fa32d1075a9174 2023-05-05 15:54:12,012 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/5d5c454854a3429ea5e9adedb86c8276 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/5d5c454854a3429ea5e9adedb86c8276 2023-05-05 15:54:12,014 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/5e04725445fd4322809d1a9d85503ec1 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/5e04725445fd4322809d1a9d85503ec1 2023-05-05 15:54:12,015 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/8593999d3ff54150bdb2dd76b2f46d3f to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/8593999d3ff54150bdb2dd76b2f46d3f 2023-05-05 15:54:12,015 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/recovered.edits/113.seqid to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8/recovered.edits/113.seqid 2023-05-05 15:54:12,015 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/recovered.edits/113.seqid to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d/recovered.edits/113.seqid 2023-05-05 15:54:12,016 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6cb2bd681dce908bd0c281b2c71d45e8 2023-05-05 15:54:12,016 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:54:12,017 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/a0e50c325aef45bd913cb3678df787a7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/a0e50c325aef45bd913cb3678df787a7 2023-05-05 15:54:12,018 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/ab3a9bf885de4438ad1f504ac52c62c2 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/ab3a9bf885de4438ad1f504ac52c62c2 2023-05-05 15:54:12,019 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/de1a5ea53f2a42578b0dcd4c71037963 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/de1a5ea53f2a42578b0dcd4c71037963 2023-05-05 15:54:12,020 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/e68abacff98449d58bd61ca4ffe85094 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/e68abacff98449d58bd61ca4ffe85094 2023-05-05 15:54:12,021 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/f87f9f855cfc4676bdef6c17cad64e1c to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf1/f87f9f855cfc4676bdef6c17cad64e1c 2023-05-05 15:54:12,024 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/119b6673ef83418c87eba55400a3fa50 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/119b6673ef83418c87eba55400a3fa50 2023-05-05 15:54:12,025 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/3d64d04b356c462c80aeb3d7341d16b5 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/3d64d04b356c462c80aeb3d7341d16b5 2023-05-05 15:54:12,026 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/5dbf1121ee934ff395d553d9c0733a84 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/5dbf1121ee934ff395d553d9c0733a84 2023-05-05 15:54:12,027 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/5dfe0faf8f924ef29b2ab0594e62a7bc to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/5dfe0faf8f924ef29b2ab0594e62a7bc 2023-05-05 15:54:12,028 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/9ae30df8bc6a4dc198df8a5bfb0a5075 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/9ae30df8bc6a4dc198df8a5bfb0a5075 2023-05-05 15:54:12,029 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/c31b04df32c4455aab59ebd1cada05ae to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/c31b04df32c4455aab59ebd1cada05ae 2023-05-05 15:54:12,030 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/dd4d733d767c4c68bdcb726c005625f4 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/dd4d733d767c4c68bdcb726c005625f4 2023-05-05 15:54:12,031 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/e7bd85555d934280bda43b21281bb981 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/e7bd85555d934280bda43b21281bb981 2023-05-05 15:54:12,032 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/f1c90388d6a64950a4d581e92c36d16a to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/f1c90388d6a64950a4d581e92c36d16a 2023-05-05 15:54:12,033 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/f775112a6fc94a92b672ce0af66e793b to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/cf2/f775112a6fc94a92b672ce0af66e793b 2023-05-05 15:54:12,036 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/recovered.edits/106.seqid to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2/recovered.edits/106.seqid 2023-05-05 15:54:12,036 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/df08d89907b6ac6a28b72ec117993ac2 2023-05-05 15:54:12,036 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-05-05 15:54:12,038 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:12,040 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-05-05 15:54:12,042 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-05-05 15:54:12,043 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:12,043 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-05-05 15:54:12,043 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302052043"}]},"ts":"9223372036854775807"} 2023-05-05 15:54:12,043 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302052043"}]},"ts":"9223372036854775807"} 2023-05-05 15:54:12,043 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302052043"}]},"ts":"9223372036854775807"} 2023-05-05 15:54:12,045 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-05 15:54:12,045 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => df08d89907b6ac6a28b72ec117993ac2, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1683302048082.df08d89907b6ac6a28b72ec117993ac2.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 6cb2bd681dce908bd0c281b2c71d45e8, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1683302050283.6cb2bd681dce908bd0c281b2c71d45e8.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => 2d9826f21eb94a559f0d5b68d98cb98d, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1683302050283.2d9826f21eb94a559f0d5b68d98cb98d.', STARTKEY => '60', ENDKEY => ''}] 2023-05-05 15:54:12,045 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-05-05 15:54:12,045 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683302052045"}]},"ts":"9223372036854775807"} 2023-05-05 15:54:12,046 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-05-05 15:54:12,048 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:54:12,049 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 52 msec 2023-05-05 15:54:12,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-05-05 15:54:12,101 INFO [Listener at localhost/35117] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-05-05 15:54:12,115 INFO [Listener at localhost/35117] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=408 (was 402) Potentially hanging thread: hconnection-0x6277894c-metaLookup-shared--pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:55052 [Waiting for operation #28] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:56120 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:44658 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6277894c-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:56204 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:44460 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6277894c-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6277894c-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:44162 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:44194 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:54532 [Waiting for operation #49] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:56516 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:56146 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:56708 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:44720 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6277894c-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6277894c-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:59852 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6277894c-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6277894c-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:59836 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6277894c-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:60356 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_741088391_15 at /127.0.0.1:56126 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6277894c-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=852 (was 839) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=299 (was 307), ProcessCount=179 (was 179), AvailableMemoryMB=5203 (was 5214) 2023-05-05 15:54:12,129 INFO [Listener at localhost/35117] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=408, OpenFileDescriptor=852, MaxFileDescriptor=60000, SystemLoadAverage=299, ProcessCount=179, AvailableMemoryMB=5203 2023-05-05 15:54:12,130 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-05 15:54:12,133 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-05 15:54:12,133 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-05 15:54:12,134 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:54:12,136 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-05-05 15:54:12,137 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-05 15:54:12,138 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0ef3937c5a4e99c77bfe5f10af1ca890 2023-05-05 15:54:12,139 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0ef3937c5a4e99c77bfe5f10af1ca890 empty. 2023-05-05 15:54:12,139 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0ef3937c5a4e99c77bfe5f10af1ca890 2023-05-05 15:54:12,139 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-05-05 15:54:12,153 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-05-05 15:54:12,154 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7675): creating {ENCODED => 0ef3937c5a4e99c77bfe5f10af1ca890, NAME => 'testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp 2023-05-05 15:54:12,166 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:54:12,167 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1604): Closing 0ef3937c5a4e99c77bfe5f10af1ca890, disabling compactions & flushes 2023-05-05 15:54:12,167 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890. 2023-05-05 15:54:12,167 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890. 2023-05-05 15:54:12,167 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890. after waiting 0 ms 2023-05-05 15:54:12,167 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890. 2023-05-05 15:54:12,167 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890. 2023-05-05 15:54:12,167 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1558): Region close journal for 0ef3937c5a4e99c77bfe5f10af1ca890: 2023-05-05 15:54:12,169 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-05-05 15:54:12,170 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683302052170"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302052170"}]},"ts":"1683302052170"} 2023-05-05 15:54:12,171 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-05 15:54:12,172 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-05 15:54:12,172 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302052172"}]},"ts":"1683302052172"} 2023-05-05 15:54:12,173 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-05-05 15:54:12,175 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-05 15:54:12,176 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-05 15:54:12,176 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-05 15:54:12,176 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-05 15:54:12,176 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-05 15:54:12,176 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0ef3937c5a4e99c77bfe5f10af1ca890, ASSIGN}] 2023-05-05 15:54:12,178 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0ef3937c5a4e99c77bfe5f10af1ca890, ASSIGN 2023-05-05 15:54:12,179 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0ef3937c5a4e99c77bfe5f10af1ca890, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40991,1683302023016; forceNewPlan=false, retain=false 2023-05-05 15:54:12,329 INFO [jenkins-hbase4:44213] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-05 15:54:12,332 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=0ef3937c5a4e99c77bfe5f10af1ca890, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:12,332 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683302052332"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302052332"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302052332"}]},"ts":"1683302052332"} 2023-05-05 15:54:12,333 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 0ef3937c5a4e99c77bfe5f10af1ca890, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:54:12,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890. 2023-05-05 15:54:12,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0ef3937c5a4e99c77bfe5f10af1ca890, NAME => 'testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890.', STARTKEY => '', ENDKEY => ''} 2023-05-05 15:54:12,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:54:12,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 0ef3937c5a4e99c77bfe5f10af1ca890 2023-05-05 15:54:12,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:54:12,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 0ef3937c5a4e99c77bfe5f10af1ca890 2023-05-05 15:54:12,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 0ef3937c5a4e99c77bfe5f10af1ca890 2023-05-05 15:54:12,490 INFO [StoreOpener-0ef3937c5a4e99c77bfe5f10af1ca890-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0ef3937c5a4e99c77bfe5f10af1ca890 2023-05-05 15:54:12,491 DEBUG [StoreOpener-0ef3937c5a4e99c77bfe5f10af1ca890-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackForSplitTableRegionWithReplica/0ef3937c5a4e99c77bfe5f10af1ca890/cf1 2023-05-05 15:54:12,491 DEBUG [StoreOpener-0ef3937c5a4e99c77bfe5f10af1ca890-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackForSplitTableRegionWithReplica/0ef3937c5a4e99c77bfe5f10af1ca890/cf1 2023-05-05 15:54:12,491 INFO [StoreOpener-0ef3937c5a4e99c77bfe5f10af1ca890-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0ef3937c5a4e99c77bfe5f10af1ca890 columnFamilyName cf1 2023-05-05 15:54:12,492 INFO [StoreOpener-0ef3937c5a4e99c77bfe5f10af1ca890-1] regionserver.HStore(310): Store=0ef3937c5a4e99c77bfe5f10af1ca890/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:54:12,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackForSplitTableRegionWithReplica/0ef3937c5a4e99c77bfe5f10af1ca890 2023-05-05 15:54:12,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackForSplitTableRegionWithReplica/0ef3937c5a4e99c77bfe5f10af1ca890 2023-05-05 15:54:12,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 0ef3937c5a4e99c77bfe5f10af1ca890 2023-05-05 15:54:12,497 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackForSplitTableRegionWithReplica/0ef3937c5a4e99c77bfe5f10af1ca890/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-05 15:54:12,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 0ef3937c5a4e99c77bfe5f10af1ca890; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60285464, jitterRate=-0.1016765832901001}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-05-05 15:54:12,497 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 0ef3937c5a4e99c77bfe5f10af1ca890: 2023-05-05 15:54:12,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890., pid=87, masterSystemTime=1683302052485 2023-05-05 15:54:12,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890. 2023-05-05 15:54:12,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890. 2023-05-05 15:54:12,500 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=0ef3937c5a4e99c77bfe5f10af1ca890, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:12,500 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683302052500"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302052500"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302052500"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302052500"}]},"ts":"1683302052500"} 2023-05-05 15:54:12,503 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-05-05 15:54:12,503 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 0ef3937c5a4e99c77bfe5f10af1ca890, server=jenkins-hbase4.apache.org,40991,1683302023016 in 168 msec 2023-05-05 15:54:12,505 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-05-05 15:54:12,505 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0ef3937c5a4e99c77bfe5f10af1ca890, ASSIGN in 327 msec 2023-05-05 15:54:12,505 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-05 15:54:12,505 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302052505"}]},"ts":"1683302052505"} 2023-05-05 15:54:12,506 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-05-05 15:54:12,513 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-05-05 15:54:12,514 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 380 msec 2023-05-05 15:54:12,644 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-05-05 15:54:12,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:54:12,663 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-05-05 15:54:12,664 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-05-05 15:54:12,664 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-05-05 15:54:12,667 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-05-05 15:54:12,681 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0ef3937c5a4e99c77bfe5f10af1ca890, REOPEN/MOVE}] 2023-05-05 15:54:12,682 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0ef3937c5a4e99c77bfe5f10af1ca890, REOPEN/MOVE 2023-05-05 15:54:12,683 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=0ef3937c5a4e99c77bfe5f10af1ca890, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:12,683 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683302052683"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302052683"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302052683"}]},"ts":"1683302052683"} 2023-05-05 15:54:12,684 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 0ef3937c5a4e99c77bfe5f10af1ca890, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:54:12,836 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 0ef3937c5a4e99c77bfe5f10af1ca890 2023-05-05 15:54:12,838 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 0ef3937c5a4e99c77bfe5f10af1ca890, disabling compactions & flushes 2023-05-05 15:54:12,838 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890. 2023-05-05 15:54:12,838 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890. 2023-05-05 15:54:12,838 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890. after waiting 0 ms 2023-05-05 15:54:12,838 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890. 2023-05-05 15:54:12,841 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackForSplitTableRegionWithReplica/0ef3937c5a4e99c77bfe5f10af1ca890/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-05-05 15:54:12,841 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:54:12,841 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890. 2023-05-05 15:54:12,841 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 0ef3937c5a4e99c77bfe5f10af1ca890: 2023-05-05 15:54:12,841 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 0ef3937c5a4e99c77bfe5f10af1ca890 to self. 2023-05-05 15:54:12,843 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 0ef3937c5a4e99c77bfe5f10af1ca890 2023-05-05 15:54:12,843 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=0ef3937c5a4e99c77bfe5f10af1ca890, regionState=CLOSED 2023-05-05 15:54:12,843 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683302052843"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302052843"}]},"ts":"1683302052843"} 2023-05-05 15:54:12,846 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-05-05 15:54:12,846 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 0ef3937c5a4e99c77bfe5f10af1ca890, server=jenkins-hbase4.apache.org,40991,1683302023016 in 161 msec 2023-05-05 15:54:12,847 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0ef3937c5a4e99c77bfe5f10af1ca890, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,40991,1683302023016; forceNewPlan=false, retain=true 2023-05-05 15:54:12,997 INFO [jenkins-hbase4:44213] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-05 15:54:12,997 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=0ef3937c5a4e99c77bfe5f10af1ca890, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:12,998 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683302052997"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302052997"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302052997"}]},"ts":"1683302052997"} 2023-05-05 15:54:12,999 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 0ef3937c5a4e99c77bfe5f10af1ca890, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:54:13,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890. 2023-05-05 15:54:13,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0ef3937c5a4e99c77bfe5f10af1ca890, NAME => 'testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890.', STARTKEY => '', ENDKEY => ''} 2023-05-05 15:54:13,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:54:13,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 0ef3937c5a4e99c77bfe5f10af1ca890 2023-05-05 15:54:13,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:54:13,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 0ef3937c5a4e99c77bfe5f10af1ca890 2023-05-05 15:54:13,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 0ef3937c5a4e99c77bfe5f10af1ca890 2023-05-05 15:54:13,155 INFO [StoreOpener-0ef3937c5a4e99c77bfe5f10af1ca890-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0ef3937c5a4e99c77bfe5f10af1ca890 2023-05-05 15:54:13,156 DEBUG [StoreOpener-0ef3937c5a4e99c77bfe5f10af1ca890-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackForSplitTableRegionWithReplica/0ef3937c5a4e99c77bfe5f10af1ca890/cf1 2023-05-05 15:54:13,156 DEBUG [StoreOpener-0ef3937c5a4e99c77bfe5f10af1ca890-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackForSplitTableRegionWithReplica/0ef3937c5a4e99c77bfe5f10af1ca890/cf1 2023-05-05 15:54:13,157 INFO [StoreOpener-0ef3937c5a4e99c77bfe5f10af1ca890-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0ef3937c5a4e99c77bfe5f10af1ca890 columnFamilyName cf1 2023-05-05 15:54:13,157 INFO [StoreOpener-0ef3937c5a4e99c77bfe5f10af1ca890-1] regionserver.HStore(310): Store=0ef3937c5a4e99c77bfe5f10af1ca890/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:54:13,158 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackForSplitTableRegionWithReplica/0ef3937c5a4e99c77bfe5f10af1ca890 2023-05-05 15:54:13,159 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackForSplitTableRegionWithReplica/0ef3937c5a4e99c77bfe5f10af1ca890 2023-05-05 15:54:13,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 0ef3937c5a4e99c77bfe5f10af1ca890 2023-05-05 15:54:13,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 0ef3937c5a4e99c77bfe5f10af1ca890; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61851171, jitterRate=-0.07834573090076447}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-05-05 15:54:13,162 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 0ef3937c5a4e99c77bfe5f10af1ca890: 2023-05-05 15:54:13,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890., pid=92, masterSystemTime=1683302053151 2023-05-05 15:54:13,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890. 2023-05-05 15:54:13,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890. 2023-05-05 15:54:13,165 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=0ef3937c5a4e99c77bfe5f10af1ca890, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:54:13,165 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683302053165"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302053165"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302053165"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302053165"}]},"ts":"1683302053165"} 2023-05-05 15:54:13,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-05-05 15:54:13,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 0ef3937c5a4e99c77bfe5f10af1ca890, server=jenkins-hbase4.apache.org,40991,1683302023016 in 167 msec 2023-05-05 15:54:13,169 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-05-05 15:54:13,169 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0ef3937c5a4e99c77bfe5f10af1ca890, REOPEN/MOVE in 487 msec 2023-05-05 15:54:13,172 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-05-05 15:54:13,172 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 504 msec 2023-05-05 15:54:13,174 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=47d3ff38207fecb2cbad8f4f5aab825b, ASSIGN}] 2023-05-05 15:54:13,176 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=47d3ff38207fecb2cbad8f4f5aab825b, ASSIGN 2023-05-05 15:54:13,176 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=47d3ff38207fecb2cbad8f4f5aab825b, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-05-05 15:54:13,327 DEBUG [jenkins-hbase4:44213] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-05 15:54:13,328 DEBUG [jenkins-hbase4:44213] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-05 15:54:13,328 DEBUG [jenkins-hbase4:44213] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-05 15:54:13,328 DEBUG [jenkins-hbase4:44213] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-05 15:54:13,328 DEBUG [jenkins-hbase4:44213] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-05-05 15:54:13,329 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=47d3ff38207fecb2cbad8f4f5aab825b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:54:13,329 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683302053329"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1683302053329"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1683302053329"}]},"ts":"1683302053329"} 2023-05-05 15:54:13,333 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 47d3ff38207fecb2cbad8f4f5aab825b, server=jenkins-hbase4.apache.org,42799,1683302022977}] 2023-05-05 15:54:13,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1683302052133_0001.47d3ff38207fecb2cbad8f4f5aab825b. 2023-05-05 15:54:13,487 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 47d3ff38207fecb2cbad8f4f5aab825b, NAME => 'testRollbackForSplitTableRegionWithReplica,,1683302052133_0001.47d3ff38207fecb2cbad8f4f5aab825b.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-05-05 15:54:13,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:54:13,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 47d3ff38207fecb2cbad8f4f5aab825b 2023-05-05 15:54:13,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1683302052133_0001.47d3ff38207fecb2cbad8f4f5aab825b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:54:13,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 47d3ff38207fecb2cbad8f4f5aab825b 2023-05-05 15:54:13,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 47d3ff38207fecb2cbad8f4f5aab825b 2023-05-05 15:54:13,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:13,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:13,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:13,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:13,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:13,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:14,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:14,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:14,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:14,341 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-05-05 15:54:14,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:14,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:14,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:14,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:14,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:14,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:14,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:15,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:15,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:15,292 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:15,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:15,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:15,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:15,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:15,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:15,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:15,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:16,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:16,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:16,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:16,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:16,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:16,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:16,632 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-05 15:54:16,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:16,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:16,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:16,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:17,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:17,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:17,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:17,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:17,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:17,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:17,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:17,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:17,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:17,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:18,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:18,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:18,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:18,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:18,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:18,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:18,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:18,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:18,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:19,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:19,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:19,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:19,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:19,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:19,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:19,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:19,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:19,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:19,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:20,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:20,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:20,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:20,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:20,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:20,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:20,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:20,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:20,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:20,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:21,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:21,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:21,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:21,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:21,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:21,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:21,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:21,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:21,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:21,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:22,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:22,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:22,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:22,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:22,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:22,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:22,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:22,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:22,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:22,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:23,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:23,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:23,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:23,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:23,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:23,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:23,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:23,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:23,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:23,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:24,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:24,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:24,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:24,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:24,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:24,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:24,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:24,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:24,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:24,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:25,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:25,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:25,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:25,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:25,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:25,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:25,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:25,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:25,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:25,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:26,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:26,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:26,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:26,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:26,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:26,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:26,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:26,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:26,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:26,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:27,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:27,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:27,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:27,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:27,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:27,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:27,621 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:27,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:27,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:27,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:28,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:28,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:28,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:28,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:28,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:28,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:28,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:28,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:28,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:28,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:29,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:29,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:29,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:29,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:29,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:29,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:29,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:29,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:29,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:29,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:30,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:30,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:30,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:30,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:30,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:30,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:30,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:30,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:30,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:30,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:31,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:31,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:31,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:31,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:31,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:31,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:31,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:31,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:31,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:31,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:32,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:32,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:32,230 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:32,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:32,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:32,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:32,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:32,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:32,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:32,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:33,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:33,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:33,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:33,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:33,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:33,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:33,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:33,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:33,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:33,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:34,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:34,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:34,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:34,334 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:34,434 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:34,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:34,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:34,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:34,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:34,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:35,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:35,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:35,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:35,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:35,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:35,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:35,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:35,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:35,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:35,937 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:36,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:36,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:36,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:36,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:36,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:36,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:36,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:36,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:36,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:36,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:37,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:37,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:37,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:37,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:37,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:37,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:37,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:37,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:37,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:37,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:38,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:38,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:38,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:38,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:38,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:38,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:38,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:38,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:38,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:38,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:39,043 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:39,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:39,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:39,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:39,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:39,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:39,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:39,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:39,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:39,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:40,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:40,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:40,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:40,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:40,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:40,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:40,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:40,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:40,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:40,947 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:41,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:41,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:41,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:41,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:41,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:41,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:41,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:41,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:41,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:41,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:42,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:42,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:42,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:42,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:42,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:42,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:42,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:42,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:42,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:42,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:43,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:43,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:43,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:43,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:43,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:43,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:43,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:43,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:43,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:43,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:44,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:44,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:44,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:44,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:44,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:44,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:44,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:44,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:44,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:44,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:45,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:45,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:45,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:45,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:45,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:45,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:45,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:45,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:45,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:45,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:46,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:46,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:46,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:46,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:46,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:46,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:46,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:46,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:46,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:46,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:47,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:47,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:47,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:47,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:47,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:47,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:47,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:47,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:47,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:47,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:48,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:48,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:48,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:48,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:48,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:48,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:48,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:48,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:48,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:48,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:49,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:49,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:49,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:49,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:49,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:49,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:49,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:49,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:49,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:49,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:50,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:50,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:50,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:50,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:50,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:50,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:50,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:50,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:50,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:50,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:51,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:51,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:51,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:51,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:51,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:51,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:51,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:51,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:51,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:51,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:52,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:52,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:52,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:52,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:52,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:52,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:52,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:52,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:52,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:52,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:53,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:53,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:53,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:53,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:53,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:53,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:53,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:53,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:53,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:53,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:54,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:54,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:54,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:54,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:54,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:54,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:54,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:54,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:54,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:54,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:55,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:55,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:55,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:55,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:55,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:55,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:55,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:55,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:55,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:55,977 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:56,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:56,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:56,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:56,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:56,478 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:56,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:56,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:56,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:56,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:56,979 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:57,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:57,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:57,280 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:57,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:57,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:57,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:57,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:57,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:57,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:57,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:58,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:58,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:58,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:58,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:58,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:58,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:58,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:58,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:58,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:58,983 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:59,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:59,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:59,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:59,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:59,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:59,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:59,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:59,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:59,885 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:54:59,985 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:00,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:00,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:00,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:00,386 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:00,486 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:00,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:00,686 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:00,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:00,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:00,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:01,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:01,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:01,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:01,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:01,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:01,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:01,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:01,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:01,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:01,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:02,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:02,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:02,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:02,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:02,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:02,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:02,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:02,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:02,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:02,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:03,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:03,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:03,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:03,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:03,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:03,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:03,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:03,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:03,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:03,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:04,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:04,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:04,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:04,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:04,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:04,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:04,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:04,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:04,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:04,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:05,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:05,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:05,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:05,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:05,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:05,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:05,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:05,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:05,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:05,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:06,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:06,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:06,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:06,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:06,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:06,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:06,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:06,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:06,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:06,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:07,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:07,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:07,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:07,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:07,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:07,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:07,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:07,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:07,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:08,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:08,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:08,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:08,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:08,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:08,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:08,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:08,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:08,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:08,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:09,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:09,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:09,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:09,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:09,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:09,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:09,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:09,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:09,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:09,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:10,005 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:10,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:10,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:10,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:10,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:10,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:10,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:10,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:10,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:10,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:11,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:11,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:11,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:11,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:11,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:11,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:11,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:11,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:11,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:11,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:12,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:12,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:12,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:12,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:12,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:12,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:12,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:12,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:12,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-05-05 15:55:12,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:12,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-05-05 15:55:12,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:13,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:13,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-05-05 15:55:13,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:13,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:13,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:13,369 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-05-05 15:55:13,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:13,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:13,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:13,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:13,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:13,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-05-05 15:55:13,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:14,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:14,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:14,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:14,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:14,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:14,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:14,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:14,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:14,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:14,871 DEBUG [Listener at localhost/35117] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:14,872 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef3937c5a4e99c77bfe5f10af1ca890, daughterA=b385b26868fcaaccb79e222957786618, daughterB=a13170d1b393d49a10002f92ce6dcd48 2023-05-05 15:55:14,873 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef3937c5a4e99c77bfe5f10af1ca890, daughterA=b385b26868fcaaccb79e222957786618, daughterB=a13170d1b393d49a10002f92ce6dcd48 2023-05-05 15:55:14,873 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef3937c5a4e99c77bfe5f10af1ca890, daughterA=b385b26868fcaaccb79e222957786618, daughterB=a13170d1b393d49a10002f92ce6dcd48 2023-05-05 15:55:14,874 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef3937c5a4e99c77bfe5f10af1ca890, daughterA=b385b26868fcaaccb79e222957786618, daughterB=a13170d1b393d49a10002f92ce6dcd48 2023-05-05 15:55:14,876 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 0ef3937c5a4e99c77bfe5f10af1ca890, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef3937c5a4e99c77bfe5f10af1ca890, daughterA=b385b26868fcaaccb79e222957786618, daughterB=a13170d1b393d49a10002f92ce6dcd48 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42799,1683302022977, table=testRollbackForSplitTableRegionWithReplica, region=47d3ff38207fecb2cbad8f4f5aab825b is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:55:14,879 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42799,1683302022977, table=testRollbackForSplitTableRegionWithReplica, region=47d3ff38207fecb2cbad8f4f5aab825b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef3937c5a4e99c77bfe5f10af1ca890, daughterA=b385b26868fcaaccb79e222957786618, daughterB=a13170d1b393d49a10002f92ce6dcd48 exec-time=6 msec 2023-05-05 15:55:14,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:15,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:15,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-05 15:55:15,216 INFO [StoreOpener-47d3ff38207fecb2cbad8f4f5aab825b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 47d3ff38207fecb2cbad8f4f5aab825b 2023-05-05 15:55:15,219 DEBUG [StoreOpener-47d3ff38207fecb2cbad8f4f5aab825b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackForSplitTableRegionWithReplica/0ef3937c5a4e99c77bfe5f10af1ca890/cf1 2023-05-05 15:55:15,219 DEBUG [StoreOpener-47d3ff38207fecb2cbad8f4f5aab825b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackForSplitTableRegionWithReplica/0ef3937c5a4e99c77bfe5f10af1ca890/cf1 2023-05-05 15:55:15,219 INFO [StoreOpener-47d3ff38207fecb2cbad8f4f5aab825b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 47d3ff38207fecb2cbad8f4f5aab825b columnFamilyName cf1 2023-05-05 15:55:15,220 INFO [StoreOpener-47d3ff38207fecb2cbad8f4f5aab825b-1] regionserver.HStore(310): Store=47d3ff38207fecb2cbad8f4f5aab825b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:55:15,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 47d3ff38207fecb2cbad8f4f5aab825b; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70442358, jitterRate=0.049672931432724}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-05-05 15:55:15,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 47d3ff38207fecb2cbad8f4f5aab825b: 2023-05-05 15:55:15,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1683302052133_0001.47d3ff38207fecb2cbad8f4f5aab825b., pid=94, masterSystemTime=1683302053484 2023-05-05 15:55:15,223 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1683302052133_0001.47d3ff38207fecb2cbad8f4f5aab825b. 2023-05-05 15:55:15,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1683302052133_0001.47d3ff38207fecb2cbad8f4f5aab825b. 2023-05-05 15:55:15,224 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=47d3ff38207fecb2cbad8f4f5aab825b, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:15,224 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683302115224"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1683302115224"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1683302115224"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1683302115224"}]},"ts":"1683302115224"} 2023-05-05 15:55:15,227 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-05-05 15:55:15,227 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 47d3ff38207fecb2cbad8f4f5aab825b, server=jenkins-hbase4.apache.org,42799,1683302022977 in 1 mins, 1.894 sec 2023-05-05 15:55:15,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-05-05 15:55:15,229 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=47d3ff38207fecb2cbad8f4f5aab825b, ASSIGN in 1 mins, 2.053 sec 2023-05-05 15:55:15,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.582 sec 2023-05-05 15:55:15,231 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1683302052641","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62589,"client":"172.31.14.131:58500","queuetimems":0,"class":"HMaster"} 2023-05-05 15:55:15,231 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 605, skipping over this response of 3 bytes 2023-05-05 15:55:17,123 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42799,1683302022977, table=testRollbackForSplitTableRegionWithReplica, region=47d3ff38207fecb2cbad8f4f5aab825b is currently in transition, give up 2023-05-05 15:55:17,125 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-05 15:55:17,126 INFO [Listener at localhost/35117] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:17,127 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:17,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:17,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-05-05 15:55:17,130 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302117130"}]},"ts":"1683302117130"} 2023-05-05 15:55:17,132 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-05-05 15:55:17,134 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-05-05 15:55:17,134 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0ef3937c5a4e99c77bfe5f10af1ca890, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=47d3ff38207fecb2cbad8f4f5aab825b, UNASSIGN}] 2023-05-05 15:55:17,135 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=47d3ff38207fecb2cbad8f4f5aab825b, UNASSIGN 2023-05-05 15:55:17,136 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=47d3ff38207fecb2cbad8f4f5aab825b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:17,136 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683302117136"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1683302117136"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1683302117136"}]},"ts":"1683302117136"} 2023-05-05 15:55:17,137 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 47d3ff38207fecb2cbad8f4f5aab825b, server=jenkins-hbase4.apache.org,42799,1683302022977}] 2023-05-05 15:55:17,139 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0ef3937c5a4e99c77bfe5f10af1ca890, UNASSIGN 2023-05-05 15:55:17,140 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=0ef3937c5a4e99c77bfe5f10af1ca890, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:17,140 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683302117140"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302117140"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302117140"}]},"ts":"1683302117140"} 2023-05-05 15:55:17,141 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 0ef3937c5a4e99c77bfe5f10af1ca890, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:55:17,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-05-05 15:55:17,289 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 47d3ff38207fecb2cbad8f4f5aab825b 2023-05-05 15:55:17,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 47d3ff38207fecb2cbad8f4f5aab825b, disabling compactions & flushes 2023-05-05 15:55:17,290 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1683302052133_0001.47d3ff38207fecb2cbad8f4f5aab825b. 2023-05-05 15:55:17,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1683302052133_0001.47d3ff38207fecb2cbad8f4f5aab825b. 2023-05-05 15:55:17,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1683302052133_0001.47d3ff38207fecb2cbad8f4f5aab825b. after waiting 0 ms 2023-05-05 15:55:17,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1683302052133_0001.47d3ff38207fecb2cbad8f4f5aab825b. 2023-05-05 15:55:17,291 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:55:17,292 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1683302052133_0001.47d3ff38207fecb2cbad8f4f5aab825b. 2023-05-05 15:55:17,292 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 47d3ff38207fecb2cbad8f4f5aab825b: 2023-05-05 15:55:17,292 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 0ef3937c5a4e99c77bfe5f10af1ca890 2023-05-05 15:55:17,293 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 0ef3937c5a4e99c77bfe5f10af1ca890, disabling compactions & flushes 2023-05-05 15:55:17,293 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890. 2023-05-05 15:55:17,293 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890. 2023-05-05 15:55:17,293 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890. after waiting 0 ms 2023-05-05 15:55:17,293 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890. 2023-05-05 15:55:17,294 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 47d3ff38207fecb2cbad8f4f5aab825b 2023-05-05 15:55:17,295 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=47d3ff38207fecb2cbad8f4f5aab825b, regionState=CLOSED 2023-05-05 15:55:17,295 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683302117294"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1683302117294"}]},"ts":"1683302117294"} 2023-05-05 15:55:17,298 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRollbackForSplitTableRegionWithReplica/0ef3937c5a4e99c77bfe5f10af1ca890/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-05-05 15:55:17,299 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-05-05 15:55:17,299 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 47d3ff38207fecb2cbad8f4f5aab825b, server=jenkins-hbase4.apache.org,42799,1683302022977 in 160 msec 2023-05-05 15:55:17,299 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:55:17,300 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890. 2023-05-05 15:55:17,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 0ef3937c5a4e99c77bfe5f10af1ca890: 2023-05-05 15:55:17,300 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=47d3ff38207fecb2cbad8f4f5aab825b, UNASSIGN in 165 msec 2023-05-05 15:55:17,301 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 0ef3937c5a4e99c77bfe5f10af1ca890 2023-05-05 15:55:17,301 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=0ef3937c5a4e99c77bfe5f10af1ca890, regionState=CLOSED 2023-05-05 15:55:17,301 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683302117301"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302117301"}]},"ts":"1683302117301"} 2023-05-05 15:55:17,304 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-05-05 15:55:17,304 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 0ef3937c5a4e99c77bfe5f10af1ca890, server=jenkins-hbase4.apache.org,40991,1683302023016 in 161 msec 2023-05-05 15:55:17,305 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-05-05 15:55:17,305 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0ef3937c5a4e99c77bfe5f10af1ca890, UNASSIGN in 170 msec 2023-05-05 15:55:17,306 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302117306"}]},"ts":"1683302117306"} 2023-05-05 15:55:17,307 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-05-05 15:55:17,309 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-05-05 15:55:17,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 182 msec 2023-05-05 15:55:17,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-05-05 15:55:17,432 INFO [Listener at localhost/35117] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-05-05 15:55:17,433 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:17,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:17,435 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:17,436 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:17,436 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-05-05 15:55:17,440 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0ef3937c5a4e99c77bfe5f10af1ca890 2023-05-05 15:55:17,442 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0ef3937c5a4e99c77bfe5f10af1ca890/cf1, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0ef3937c5a4e99c77bfe5f10af1ca890/recovered.edits] 2023-05-05 15:55:17,446 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0ef3937c5a4e99c77bfe5f10af1ca890/recovered.edits/7.seqid to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRollbackForSplitTableRegionWithReplica/0ef3937c5a4e99c77bfe5f10af1ca890/recovered.edits/7.seqid 2023-05-05 15:55:17,446 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0ef3937c5a4e99c77bfe5f10af1ca890 2023-05-05 15:55:17,446 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-05-05 15:55:17,449 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:17,451 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-05-05 15:55:17,453 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-05-05 15:55:17,454 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:17,454 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-05-05 15:55:17,454 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302117454"}]},"ts":"9223372036854775807"} 2023-05-05 15:55:17,454 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1683302052133_0001.47d3ff38207fecb2cbad8f4f5aab825b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302117454"}]},"ts":"9223372036854775807"} 2023-05-05 15:55:17,456 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-05-05 15:55:17,456 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 0ef3937c5a4e99c77bfe5f10af1ca890, NAME => 'testRollbackForSplitTableRegionWithReplica,,1683302052133.0ef3937c5a4e99c77bfe5f10af1ca890.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 47d3ff38207fecb2cbad8f4f5aab825b, NAME => 'testRollbackForSplitTableRegionWithReplica,,1683302052133_0001.47d3ff38207fecb2cbad8f4f5aab825b.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-05-05 15:55:17,456 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-05-05 15:55:17,456 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683302117456"}]},"ts":"9223372036854775807"} 2023-05-05 15:55:17,458 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-05-05 15:55:17,459 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:17,460 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 27 msec 2023-05-05 15:55:17,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-05-05 15:55:17,537 INFO [Listener at localhost/35117] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-05-05 15:55:17,550 INFO [Listener at localhost/35117] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=343 (was 408), OpenFileDescriptor=653 (was 852), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=167 (was 299), ProcessCount=179 (was 179), AvailableMemoryMB=5183 (was 5203) 2023-05-05 15:55:17,564 INFO [Listener at localhost/35117] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=344, OpenFileDescriptor=653, MaxFileDescriptor=60000, SystemLoadAverage=167, ProcessCount=179, AvailableMemoryMB=5183 2023-05-05 15:55:17,564 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-05 15:55:17,567 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-05 15:55:17,568 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-05 15:55:17,569 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:17,570 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-05-05 15:55:17,571 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-05 15:55:17,573 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c 2023-05-05 15:55:17,573 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c empty. 2023-05-05 15:55:17,574 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c 2023-05-05 15:55:17,574 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-05-05 15:55:17,583 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-05-05 15:55:17,585 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7675): creating {ENCODED => 96da58919e3f0f706f77338d9b98be4c, NAME => 'testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp 2023-05-05 15:55:17,591 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:55:17,591 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1604): Closing 96da58919e3f0f706f77338d9b98be4c, disabling compactions & flushes 2023-05-05 15:55:17,591 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. 2023-05-05 15:55:17,591 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. 2023-05-05 15:55:17,591 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. after waiting 0 ms 2023-05-05 15:55:17,591 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. 2023-05-05 15:55:17,591 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. 2023-05-05 15:55:17,591 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1558): Region close journal for 96da58919e3f0f706f77338d9b98be4c: 2023-05-05 15:55:17,593 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-05-05 15:55:17,594 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1683302117594"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302117594"}]},"ts":"1683302117594"} 2023-05-05 15:55:17,595 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-05 15:55:17,596 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-05 15:55:17,596 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302117596"}]},"ts":"1683302117596"} 2023-05-05 15:55:17,597 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-05-05 15:55:17,599 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-05 15:55:17,601 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-05 15:55:17,601 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-05 15:55:17,601 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-05 15:55:17,601 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-05 15:55:17,601 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=96da58919e3f0f706f77338d9b98be4c, ASSIGN}] 2023-05-05 15:55:17,602 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=96da58919e3f0f706f77338d9b98be4c, ASSIGN 2023-05-05 15:55:17,603 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=96da58919e3f0f706f77338d9b98be4c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43023,1683302022922; forceNewPlan=false, retain=false 2023-05-05 15:55:17,753 INFO [jenkins-hbase4:44213] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-05 15:55:17,754 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=96da58919e3f0f706f77338d9b98be4c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:17,754 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1683302117754"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302117754"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302117754"}]},"ts":"1683302117754"} 2023-05-05 15:55:17,756 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure 96da58919e3f0f706f77338d9b98be4c, server=jenkins-hbase4.apache.org,43023,1683302022922}] 2023-05-05 15:55:17,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. 2023-05-05 15:55:17,912 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 96da58919e3f0f706f77338d9b98be4c, NAME => 'testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c.', STARTKEY => '', ENDKEY => ''} 2023-05-05 15:55:17,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:55:17,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey 96da58919e3f0f706f77338d9b98be4c 2023-05-05 15:55:17,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:55:17,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 96da58919e3f0f706f77338d9b98be4c 2023-05-05 15:55:17,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 96da58919e3f0f706f77338d9b98be4c 2023-05-05 15:55:17,914 INFO [StoreOpener-96da58919e3f0f706f77338d9b98be4c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 96da58919e3f0f706f77338d9b98be4c 2023-05-05 15:55:17,915 DEBUG [StoreOpener-96da58919e3f0f706f77338d9b98be4c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1 2023-05-05 15:55:17,915 DEBUG [StoreOpener-96da58919e3f0f706f77338d9b98be4c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1 2023-05-05 15:55:17,915 INFO [StoreOpener-96da58919e3f0f706f77338d9b98be4c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 96da58919e3f0f706f77338d9b98be4c columnFamilyName cf1 2023-05-05 15:55:17,916 INFO [StoreOpener-96da58919e3f0f706f77338d9b98be4c-1] regionserver.HStore(310): Store=96da58919e3f0f706f77338d9b98be4c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:55:17,916 INFO [StoreOpener-96da58919e3f0f706f77338d9b98be4c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 96da58919e3f0f706f77338d9b98be4c 2023-05-05 15:55:17,917 DEBUG [StoreOpener-96da58919e3f0f706f77338d9b98be4c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2 2023-05-05 15:55:17,917 DEBUG [StoreOpener-96da58919e3f0f706f77338d9b98be4c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2 2023-05-05 15:55:17,917 INFO [StoreOpener-96da58919e3f0f706f77338d9b98be4c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 96da58919e3f0f706f77338d9b98be4c columnFamilyName cf2 2023-05-05 15:55:17,918 INFO [StoreOpener-96da58919e3f0f706f77338d9b98be4c-1] regionserver.HStore(310): Store=96da58919e3f0f706f77338d9b98be4c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:55:17,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c 2023-05-05 15:55:17,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c 2023-05-05 15:55:17,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-05 15:55:17,921 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 96da58919e3f0f706f77338d9b98be4c 2023-05-05 15:55:17,923 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-05 15:55:17,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 96da58919e3f0f706f77338d9b98be4c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65622754, jitterRate=-0.02214476466178894}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-05 15:55:17,923 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 96da58919e3f0f706f77338d9b98be4c: 2023-05-05 15:55:17,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c., pid=104, masterSystemTime=1683302117907 2023-05-05 15:55:17,926 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. 2023-05-05 15:55:17,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. 2023-05-05 15:55:17,926 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=96da58919e3f0f706f77338d9b98be4c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:17,926 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1683302117926"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302117926"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302117926"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302117926"}]},"ts":"1683302117926"} 2023-05-05 15:55:17,929 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-05-05 15:55:17,929 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure 96da58919e3f0f706f77338d9b98be4c, server=jenkins-hbase4.apache.org,43023,1683302022922 in 172 msec 2023-05-05 15:55:17,931 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-05-05 15:55:17,931 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=96da58919e3f0f706f77338d9b98be4c, ASSIGN in 328 msec 2023-05-05 15:55:17,932 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-05 15:55:17,932 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302117932"}]},"ts":"1683302117932"} 2023-05-05 15:55:17,933 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-05-05 15:55:17,935 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-05-05 15:55:17,936 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 367 msec 2023-05-05 15:55:18,071 DEBUG [Listener at localhost/35117] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-05-05 15:55:18,074 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58214, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-05-05 15:55:18,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:55:18,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-05-05 15:55:18,083 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-05-05 15:55:18,084 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:18,084 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-05-05 15:55:18,084 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-05-05 15:55:18,084 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,084 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,087 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,087 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,087 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,087 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,087 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,087 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:55:18,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,088 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,088 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,088 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:18,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,088 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-05 15:55:18,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:18,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:18,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,088 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-05 15:55:18,088 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-05 15:55:18,088 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-05 15:55:18,088 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-05 15:55:18,088 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:18,089 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-05 15:55:18,089 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-05 15:55:18,089 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-05 15:55:18,089 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-05 15:55:18,089 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-05 15:55:18,089 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-05 15:55:18,089 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-05 15:55:18,089 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:18,089 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:18,089 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-05 15:55:18,089 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-05 15:55:18,089 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-05 15:55:18,089 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-05 15:55:18,089 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:55:18,089 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:55:18,089 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. 2023-05-05 15:55:18,089 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. started... 2023-05-05 15:55:18,090 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 96da58919e3f0f706f77338d9b98be4c 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-05 15:55:18,090 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,091 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,092 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,092 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:18,092 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,092 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:18,102 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf1/24ab9032678f47018b3237626408a42b 2023-05-05 15:55:18,123 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf2/0147a67221af41ecb3e465723c40ea1a 2023-05-05 15:55:18,128 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf1/24ab9032678f47018b3237626408a42b as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/24ab9032678f47018b3237626408a42b 2023-05-05 15:55:18,132 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/24ab9032678f47018b3237626408a42b, entries=2, sequenceid=6, filesize=4.8 K 2023-05-05 15:55:18,132 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf2/0147a67221af41ecb3e465723c40ea1a as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/0147a67221af41ecb3e465723c40ea1a 2023-05-05 15:55:18,137 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/0147a67221af41ecb3e465723c40ea1a, entries=2, sequenceid=6, filesize=4.8 K 2023-05-05 15:55:18,138 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 96da58919e3f0f706f77338d9b98be4c in 48ms, sequenceid=6, compaction requested=false 2023-05-05 15:55:18,138 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-05-05 15:55:18,138 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 96da58919e3f0f706f77338d9b98be4c: 2023-05-05 15:55:18,138 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. 2023-05-05 15:55:18,138 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:55:18,138 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:55:18,138 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,138 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-05 15:55:18,138 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-05 15:55:18,140 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,140 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:18,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:18,141 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,141 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:18,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:18,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:18,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:18,142 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-05-05 15:55:18,142 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6950a137[Count = 0] remaining members to acquire global barrier 2023-05-05 15:55:18,142 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-05-05 15:55:18,143 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,145 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,145 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,145 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,145 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-05 15:55:18,145 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-05 15:55:18,145 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:55:18,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,145 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-05 15:55:18,145 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-05 15:55:18,145 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,145 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:55:18,145 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-05 15:55:18,145 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:55:18,145 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-05 15:55:18,145 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:55:18,147 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-05 15:55:18,147 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:18,147 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,147 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-05 15:55:18,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:18,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:18,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:18,148 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-05 15:55:18,148 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:18,148 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-05 15:55:18,148 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:18,148 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-05 15:55:18,148 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-05 15:55:18,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:18,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:18,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,43023,1683302022922': 2023-05-05 15:55:18,151 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-05-05 15:55:18,151 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-05-05 15:55:18,151 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:55:18,151 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:55:18,151 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-05-05 15:55:18,151 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:55:18,153 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,153 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,153 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,153 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,153 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,153 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,153 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:18,153 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:18,153 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:18,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,154 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:18,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,155 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,156 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,156 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:18,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,160 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,160 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,160 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,160 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,160 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,160 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:18,160 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,160 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,160 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,160 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-05-05 15:55:18,161 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:55:18,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,161 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testInvalidSplitKey because node does not exist (not an error) 2023-05-05 15:55:18,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,160 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,160 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,161 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,161 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,161 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-05-05 15:55:18,161 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:55:18,161 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,161 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,161 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,161 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,161 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,261 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:55:18,262 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:55:18,274 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:55:18,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-05-05 15:55:18,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-05-05 15:55:18,275 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-05-05 15:55:18,275 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:18,275 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-05-05 15:55:18,275 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-05-05 15:55:18,276 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,276 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,277 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,277 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,277 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,277 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,278 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,278 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:55:18,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,278 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,278 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,278 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:18,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:18,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:18,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,278 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-05 15:55:18,279 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-05 15:55:18,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,279 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-05 15:55:18,279 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-05 15:55:18,279 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-05 15:55:18,279 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-05 15:55:18,279 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-05 15:55:18,279 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-05 15:55:18,279 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-05 15:55:18,279 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-05 15:55:18,279 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:18,279 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-05 15:55:18,279 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-05 15:55:18,279 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-05 15:55:18,279 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:18,279 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:18,279 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-05 15:55:18,279 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-05 15:55:18,279 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-05 15:55:18,279 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-05 15:55:18,279 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-05 15:55:18,279 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-05 15:55:18,280 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:55:18,280 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:55:18,280 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. 2023-05-05 15:55:18,280 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. started... 2023-05-05 15:55:18,281 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 96da58919e3f0f706f77338d9b98be4c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:55:18,284 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,284 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,284 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,284 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,284 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:18,284 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:18,295 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf1/f2521f29013344a4a467a518b217f301 2023-05-05 15:55:18,310 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf2/4f08ef299f5a4157b94dc10f9bea62c4 2023-05-05 15:55:18,315 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf1/f2521f29013344a4a467a518b217f301 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/f2521f29013344a4a467a518b217f301 2023-05-05 15:55:18,318 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/f2521f29013344a4a467a518b217f301, entries=10, sequenceid=19, filesize=5.0 K 2023-05-05 15:55:18,319 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf2/4f08ef299f5a4157b94dc10f9bea62c4 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/4f08ef299f5a4157b94dc10f9bea62c4 2023-05-05 15:55:18,322 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/4f08ef299f5a4157b94dc10f9bea62c4, entries=10, sequenceid=19, filesize=5.0 K 2023-05-05 15:55:18,323 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 96da58919e3f0f706f77338d9b98be4c in 42ms, sequenceid=19, compaction requested=false 2023-05-05 15:55:18,323 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 96da58919e3f0f706f77338d9b98be4c: 2023-05-05 15:55:18,323 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. 2023-05-05 15:55:18,324 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:55:18,324 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:55:18,324 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,324 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-05 15:55:18,324 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-05 15:55:18,326 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,326 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:18,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:18,327 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,327 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:18,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:18,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:18,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:18,329 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-05-05 15:55:18,329 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@f23f8f2[Count = 0] remaining members to acquire global barrier 2023-05-05 15:55:18,329 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-05-05 15:55:18,329 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,330 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,330 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,330 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,330 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-05 15:55:18,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,330 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-05 15:55:18,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,330 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:55:18,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,330 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-05 15:55:18,331 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,331 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-05 15:55:18,331 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:55:18,331 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-05 15:55:18,331 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-05 15:55:18,331 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:55:18,331 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:55:18,332 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-05 15:55:18,332 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:18,332 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-05 15:55:18,333 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-05 15:55:18,333 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-05 15:55:18,333 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:18,333 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,333 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-05 15:55:18,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:18,333 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:18,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:18,333 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-05 15:55:18,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:18,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:18,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:18,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,43023,1683302022922': 2023-05-05 15:55:18,337 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-05-05 15:55:18,337 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-05-05 15:55:18,337 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:55:18,337 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:55:18,337 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-05-05 15:55:18,337 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:55:18,339 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,339 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,339 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,339 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,339 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,340 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:18,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:18,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,340 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,340 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:18,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,341 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,341 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:18,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:18,343 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,343 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,343 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,346 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,346 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,346 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,346 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,346 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,346 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,346 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,346 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,347 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-05-05 15:55:18,347 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:55:18,346 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,347 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:18,347 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,347 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-05-05 15:55:18,347 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,347 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,347 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,347 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,347 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:55:18,348 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,348 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,348 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,448 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:55:18,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:55:18,462 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:55:18,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-05-05 15:55:18,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-05-05 15:55:18,463 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-05-05 15:55:18,463 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:18,463 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-05-05 15:55:18,463 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-05-05 15:55:18,464 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,464 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,465 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,465 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,465 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,465 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,465 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,466 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:55:18,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,466 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,466 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,466 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:18,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,466 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-05 15:55:18,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:18,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,466 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-05 15:55:18,467 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-05 15:55:18,467 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-05 15:55:18,467 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-05 15:55:18,467 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-05 15:55:18,467 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:18,467 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-05 15:55:18,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:18,467 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-05 15:55:18,467 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:18,467 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-05 15:55:18,467 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-05 15:55:18,467 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-05 15:55:18,467 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-05 15:55:18,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,467 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-05 15:55:18,467 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:55:18,467 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:55:18,467 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. 2023-05-05 15:55:18,467 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-05 15:55:18,467 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. started... 2023-05-05 15:55:18,467 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-05 15:55:18,468 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-05 15:55:18,468 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:18,468 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 96da58919e3f0f706f77338d9b98be4c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:55:18,468 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-05 15:55:18,468 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-05 15:55:18,468 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-05 15:55:18,468 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,471 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,471 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,471 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:18,472 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,472 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:18,478 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf1/a162e14ba5ff4b3fa56d5392e8070946 2023-05-05 15:55:18,490 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf2/c73fe31235fb404f8ae35a1fe512a8d6 2023-05-05 15:55:18,494 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf1/a162e14ba5ff4b3fa56d5392e8070946 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/a162e14ba5ff4b3fa56d5392e8070946 2023-05-05 15:55:18,497 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/a162e14ba5ff4b3fa56d5392e8070946, entries=10, sequenceid=32, filesize=5.0 K 2023-05-05 15:55:18,498 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf2/c73fe31235fb404f8ae35a1fe512a8d6 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/c73fe31235fb404f8ae35a1fe512a8d6 2023-05-05 15:55:18,502 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/c73fe31235fb404f8ae35a1fe512a8d6, entries=10, sequenceid=32, filesize=5.0 K 2023-05-05 15:55:18,503 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 96da58919e3f0f706f77338d9b98be4c in 35ms, sequenceid=32, compaction requested=true 2023-05-05 15:55:18,503 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 96da58919e3f0f706f77338d9b98be4c: 2023-05-05 15:55:18,503 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. 2023-05-05 15:55:18,503 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:55:18,503 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:55:18,503 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,503 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-05 15:55:18,503 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-05 15:55:18,505 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,505 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:18,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:18,505 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,505 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:18,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:18,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:18,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:18,507 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-05-05 15:55:18,507 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3d28ff31[Count = 0] remaining members to acquire global barrier 2023-05-05 15:55:18,507 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-05-05 15:55:18,507 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,508 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,508 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,508 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,508 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,508 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:55:18,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,508 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-05 15:55:18,508 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-05 15:55:18,509 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-05 15:55:18,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,509 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:55:18,509 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-05 15:55:18,509 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-05 15:55:18,508 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-05 15:55:18,509 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:55:18,509 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:55:18,510 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-05 15:55:18,510 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,510 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:18,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:18,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:18,510 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-05 15:55:18,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:18,512 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-05 15:55:18,512 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:18,512 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-05 15:55:18,512 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:18,512 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-05 15:55:18,512 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-05 15:55:18,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:18,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:18,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,43023,1683302022922': 2023-05-05 15:55:18,515 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-05-05 15:55:18,515 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-05-05 15:55:18,515 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:55:18,515 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:55:18,515 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-05-05 15:55:18,515 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:55:18,516 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,517 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,517 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,517 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,517 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,516 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,517 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:18,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,517 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:18,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,518 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:18,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,518 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:18,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,519 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:18,520 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,520 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,525 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,525 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,525 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,525 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,525 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,525 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,525 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,525 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,525 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,525 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-05-05 15:55:18,526 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:55:18,525 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:18,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,526 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-05-05 15:55:18,526 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,526 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:55:18,526 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,526 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,526 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,526 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,526 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,526 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,527 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,626 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:55:18,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:55:18,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:55:18,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-05-05 15:55:18,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-05-05 15:55:18,641 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-05-05 15:55:18,641 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:18,641 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-05-05 15:55:18,641 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-05-05 15:55:18,641 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,641 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,643 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,643 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,643 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,643 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,644 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,644 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:55:18,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,644 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,645 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,645 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:18,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,645 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-05 15:55:18,645 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-05 15:55:18,645 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-05 15:55:18,645 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-05 15:55:18,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:18,645 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:18,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:18,645 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-05 15:55:18,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,645 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-05 15:55:18,645 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:55:18,645 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. 2023-05-05 15:55:18,645 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-05 15:55:18,646 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-05 15:55:18,646 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. started... 2023-05-05 15:55:18,646 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-05 15:55:18,645 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:55:18,646 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-05 15:55:18,646 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 96da58919e3f0f706f77338d9b98be4c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:55:18,646 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:18,646 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-05 15:55:18,646 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-05 15:55:18,646 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-05 15:55:18,646 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-05 15:55:18,646 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:18,646 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-05 15:55:18,646 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-05 15:55:18,646 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-05 15:55:18,646 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-05 15:55:18,646 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-05 15:55:18,648 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,648 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,648 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,648 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:18,648 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,648 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:18,658 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf1/cf4529a05f584439911b641412e926b6 2023-05-05 15:55:18,674 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf2/cdd7c4a5b9c84f7f8c0c687c1a98051e 2023-05-05 15:55:18,679 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf1/cf4529a05f584439911b641412e926b6 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/cf4529a05f584439911b641412e926b6 2023-05-05 15:55:18,684 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/cf4529a05f584439911b641412e926b6, entries=10, sequenceid=45, filesize=5.0 K 2023-05-05 15:55:18,685 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf2/cdd7c4a5b9c84f7f8c0c687c1a98051e as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/cdd7c4a5b9c84f7f8c0c687c1a98051e 2023-05-05 15:55:18,689 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/cdd7c4a5b9c84f7f8c0c687c1a98051e, entries=10, sequenceid=45, filesize=5.0 K 2023-05-05 15:55:18,690 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 96da58919e3f0f706f77338d9b98be4c in 44ms, sequenceid=45, compaction requested=true 2023-05-05 15:55:18,690 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 96da58919e3f0f706f77338d9b98be4c: 2023-05-05 15:55:18,690 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. 2023-05-05 15:55:18,691 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:55:18,691 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:55:18,691 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,691 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-05 15:55:18,691 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-05 15:55:18,693 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:18,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:18,693 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,693 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:18,693 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:18,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:18,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:18,695 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-05-05 15:55:18,695 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@50f0f7[Count = 0] remaining members to acquire global barrier 2023-05-05 15:55:18,695 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-05-05 15:55:18,695 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,696 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,696 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,696 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,697 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,697 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:55:18,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,697 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-05 15:55:18,697 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-05 15:55:18,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,697 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:55:18,697 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-05 15:55:18,697 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-05 15:55:18,697 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:55:18,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,697 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-05 15:55:18,697 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-05 15:55:18,697 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:55:18,699 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,699 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-05 15:55:18,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:18,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:18,699 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:18,700 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-05 15:55:18,700 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-05 15:55:18,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:18,700 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-05 15:55:18,700 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:18,700 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:18,700 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-05 15:55:18,700 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-05 15:55:18,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:18,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:18,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,43023,1683302022922': 2023-05-05 15:55:18,703 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-05-05 15:55:18,703 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-05-05 15:55:18,703 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:55:18,703 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:55:18,703 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-05-05 15:55:18,703 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:55:18,706 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,706 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,706 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:18,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:18,706 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,706 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,706 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,706 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:18,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,707 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,707 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:18,708 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:18,709 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,710 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,710 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,712 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,713 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,713 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,713 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,713 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,713 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,713 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-05-05 15:55:18,713 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:55:18,713 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,713 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,713 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,713 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,713 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,713 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-05-05 15:55:18,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,713 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:18,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,713 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:55:18,713 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,714 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,714 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,714 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,714 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,714 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,814 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:55:18,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:55:18,826 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:55:18,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-05-05 15:55:18,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-05-05 15:55:18,828 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-05-05 15:55:18,828 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:18,828 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-05-05 15:55:18,828 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-05-05 15:55:18,828 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,828 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,831 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,831 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,831 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,831 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,831 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,831 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:55:18,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,832 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,832 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:18,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,832 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,833 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-05 15:55:18,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:18,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,833 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-05 15:55:18,833 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-05 15:55:18,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:18,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,833 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-05 15:55:18,833 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-05 15:55:18,833 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-05 15:55:18,833 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-05 15:55:18,833 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-05 15:55:18,833 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:18,833 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-05 15:55:18,834 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-05 15:55:18,834 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-05 15:55:18,834 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:18,834 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-05 15:55:18,833 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-05 15:55:18,834 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-05 15:55:18,834 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-05 15:55:18,834 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-05 15:55:18,834 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:18,834 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-05 15:55:18,834 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-05 15:55:18,834 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-05 15:55:18,834 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:55:18,834 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:55:18,834 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. 2023-05-05 15:55:18,834 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. started... 2023-05-05 15:55:18,834 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 96da58919e3f0f706f77338d9b98be4c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:55:18,835 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,836 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,836 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,836 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:18,837 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,837 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:18,846 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf1/950ac5a1b2ba426cbfa922fc922b3bba 2023-05-05 15:55:18,858 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf2/80ac8790a8154ccdb60a432b05abcd28 2023-05-05 15:55:18,862 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf1/950ac5a1b2ba426cbfa922fc922b3bba as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/950ac5a1b2ba426cbfa922fc922b3bba 2023-05-05 15:55:18,866 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/950ac5a1b2ba426cbfa922fc922b3bba, entries=10, sequenceid=58, filesize=5.0 K 2023-05-05 15:55:18,867 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf2/80ac8790a8154ccdb60a432b05abcd28 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/80ac8790a8154ccdb60a432b05abcd28 2023-05-05 15:55:18,870 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/80ac8790a8154ccdb60a432b05abcd28, entries=10, sequenceid=58, filesize=5.0 K 2023-05-05 15:55:18,871 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 96da58919e3f0f706f77338d9b98be4c in 37ms, sequenceid=58, compaction requested=true 2023-05-05 15:55:18,871 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 96da58919e3f0f706f77338d9b98be4c: 2023-05-05 15:55:18,871 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. 2023-05-05 15:55:18,872 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:55:18,872 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:55:18,872 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,872 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-05 15:55:18,872 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-05 15:55:18,873 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,873 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:18,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:18,873 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,874 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:18,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:18,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:18,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:18,875 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-05-05 15:55:18,876 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6c65b0d3[Count = 0] remaining members to acquire global barrier 2023-05-05 15:55:18,876 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-05-05 15:55:18,876 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,878 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,878 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,878 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,878 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,878 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-05 15:55:18,878 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:55:18,878 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-05 15:55:18,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,878 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-05 15:55:18,878 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-05 15:55:18,878 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-05 15:55:18,878 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-05 15:55:18,878 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:55:18,878 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:55:18,878 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:55:18,882 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,882 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-05 15:55:18,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:18,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:18,882 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:18,882 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-05 15:55:18,882 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-05 15:55:18,883 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:18,883 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-05 15:55:18,883 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-05 15:55:18,883 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:18,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:18,883 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-05 15:55:18,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:18,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:18,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,43023,1683302022922': 2023-05-05 15:55:18,886 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-05-05 15:55:18,886 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-05-05 15:55:18,886 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:55:18,886 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:55:18,886 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-05-05 15:55:18,886 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:55:18,890 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,890 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,890 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,890 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:18,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:18,890 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,890 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,890 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:18,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,891 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,891 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:18,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:18,892 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:18,893 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:18,893 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,894 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,903 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,903 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,903 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,903 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,903 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:18,903 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,903 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:18,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,903 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,904 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:18,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:18,903 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,903 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-05-05 15:55:18,904 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:55:18,904 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:18,904 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:18,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,904 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:18,904 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-05-05 15:55:18,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:18,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:18,904 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,904 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:55:18,905 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:18,905 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:18,905 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:18,905 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:19,005 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:55:19,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:55:19,016 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:55:19,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-05-05 15:55:19,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-05-05 15:55:19,018 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-05-05 15:55:19,018 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:19,018 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-05-05 15:55:19,018 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-05-05 15:55:19,018 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:19,018 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:19,020 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:19,020 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:19,020 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:19,020 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:19,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:19,020 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:19,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:19,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:19,020 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:55:19,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:19,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:19,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:19,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:19,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:19,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:19,021 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:19,021 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:19,021 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:19,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:19,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:19,021 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-05 15:55:19,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:19,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:19,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:19,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:19,021 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-05 15:55:19,021 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-05 15:55:19,021 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-05 15:55:19,021 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-05 15:55:19,021 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-05 15:55:19,021 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-05 15:55:19,021 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-05 15:55:19,021 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:19,022 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-05 15:55:19,022 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-05 15:55:19,022 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-05 15:55:19,021 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-05 15:55:19,022 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-05 15:55:19,022 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-05 15:55:19,022 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-05 15:55:19,022 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:19,022 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:19,022 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-05 15:55:19,022 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-05 15:55:19,022 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-05 15:55:19,022 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-05 15:55:19,022 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:55:19,022 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:55:19,022 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. 2023-05-05 15:55:19,022 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. started... 2023-05-05 15:55:19,022 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 96da58919e3f0f706f77338d9b98be4c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:55:19,023 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:19,024 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:19,024 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:19,024 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:19,024 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:19,024 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:19,031 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf1/99e59478f71a4ea884924bdf39dfdbe0 2023-05-05 15:55:19,043 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf2/4b7dc063029145a480512fc5dfcf3fa4 2023-05-05 15:55:19,046 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf1/99e59478f71a4ea884924bdf39dfdbe0 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/99e59478f71a4ea884924bdf39dfdbe0 2023-05-05 15:55:19,051 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/99e59478f71a4ea884924bdf39dfdbe0, entries=10, sequenceid=71, filesize=5.0 K 2023-05-05 15:55:19,052 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf2/4b7dc063029145a480512fc5dfcf3fa4 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/4b7dc063029145a480512fc5dfcf3fa4 2023-05-05 15:55:19,056 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/4b7dc063029145a480512fc5dfcf3fa4, entries=10, sequenceid=71, filesize=5.0 K 2023-05-05 15:55:19,057 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 96da58919e3f0f706f77338d9b98be4c in 35ms, sequenceid=71, compaction requested=true 2023-05-05 15:55:19,057 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 96da58919e3f0f706f77338d9b98be4c: 2023-05-05 15:55:19,057 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. 2023-05-05 15:55:19,057 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:55:19,057 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:55:19,057 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:19,057 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-05 15:55:19,057 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-05 15:55:19,059 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:19,059 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:19,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:19,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:19,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:19,060 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:19,060 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:19,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:19,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:19,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:19,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:19,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:19,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:19,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:19,061 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-05-05 15:55:19,061 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7c9131c5[Count = 0] remaining members to acquire global barrier 2023-05-05 15:55:19,061 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-05-05 15:55:19,062 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:19,062 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:19,063 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:19,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:19,063 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:19,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:19,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:19,063 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-05 15:55:19,063 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-05 15:55:19,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:19,063 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:55:19,063 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:19,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:19,063 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:55:19,063 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-05 15:55:19,063 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-05 15:55:19,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:19,063 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:55:19,063 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-05 15:55:19,063 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-05 15:55:19,063 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:55:19,064 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:19,064 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-05 15:55:19,064 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:19,064 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-05 15:55:19,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:19,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:19,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:19,065 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-05 15:55:19,065 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:19,065 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-05 15:55:19,067 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-05 15:55:19,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:19,067 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:19,067 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-05 15:55:19,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:19,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:19,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:19,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:19,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:19,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:19,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:19,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:19,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:19,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:19,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,43023,1683302022922': 2023-05-05 15:55:19,070 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-05-05 15:55:19,070 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-05-05 15:55:19,070 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:55:19,070 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:55:19,070 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-05-05 15:55:19,070 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:55:19,072 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:19,072 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:19,072 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:19,072 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:19,072 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:19,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:19,072 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:19,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:19,072 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:19,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:19,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:19,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:19,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:19,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:19,072 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:19,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:19,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:19,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:19,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:19,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:19,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:19,073 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:19,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:19,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:19,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:19,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:19,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:19,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:19,073 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:19,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:19,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:19,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:19,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:19,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:19,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:19,075 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:19,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:19,075 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:19,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-05 15:55:19,075 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:19,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:19,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:19,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:19,078 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:19,078 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:19,078 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:19,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:19,078 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:19,078 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:19,078 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-05-05 15:55:19,078 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:55:19,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:19,078 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:19,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:19,078 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:19,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:19,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:19,078 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-05-05 15:55:19,078 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:19,079 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:55:19,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:19,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:19,078 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:19,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:19,079 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:19,079 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:19,079 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-05 15:55:19,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:19,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:19,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:19,079 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:19,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:19,079 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:19,079 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:19,079 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:19,079 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-05 15:55:19,079 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-05 15:55:19,179 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:55:19,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:55:19,186 INFO [Listener at localhost/35117] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:19,187 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43023] regionserver.StoreUtils(129): cannot split hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/f2521f29013344a4a467a518b217f301 because midkey is the same as first or last row 2023-05-05 15:55:19,187 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43023] regionserver.StoreUtils(129): cannot split hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/4f08ef299f5a4157b94dc10f9bea62c4 because midkey is the same as first or last row 2023-05-05 15:55:19,188 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43023] regionserver.HRegion(2745): Flushing 96da58919e3f0f706f77338d9b98be4c 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-05 15:55:19,226 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43023] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf1/8903832a1daa48b1980047d77680178c 2023-05-05 15:55:19,238 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43023] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf2/52714d52ed3d41a7be4312b9b813bbd6 2023-05-05 15:55:19,242 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43023] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf1/8903832a1daa48b1980047d77680178c as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/8903832a1daa48b1980047d77680178c 2023-05-05 15:55:19,245 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43023] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/8903832a1daa48b1980047d77680178c, entries=8, sequenceid=82, filesize=5.0 K 2023-05-05 15:55:19,246 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43023] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/.tmp/cf2/52714d52ed3d41a7be4312b9b813bbd6 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/52714d52ed3d41a7be4312b9b813bbd6 2023-05-05 15:55:19,249 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43023] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/52714d52ed3d41a7be4312b9b813bbd6, entries=8, sequenceid=82, filesize=5.0 K 2023-05-05 15:55:19,250 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43023] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 96da58919e3f0f706f77338d9b98be4c in 63ms, sequenceid=82, compaction requested=true 2023-05-05 15:55:19,251 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43023] regionserver.HRegion(2446): Flush status journal for 96da58919e3f0f706f77338d9b98be4c: 2023-05-05 15:55:19,251 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43023] regionserver.StoreUtils(129): cannot split hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/f2521f29013344a4a467a518b217f301 because midkey is the same as first or last row 2023-05-05 15:55:19,251 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43023] regionserver.StoreUtils(129): cannot split hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/4f08ef299f5a4157b94dc10f9bea62c4 because midkey is the same as first or last row 2023-05-05 15:55:19,252 DEBUG [Listener at localhost/35117] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:19,252 DEBUG [Listener at localhost/35117] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-05-05 15:55:19,252 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-05 15:55:19,253 INFO [Listener at localhost/35117] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-05-05 15:55:19,253 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testInvalidSplitKey 2023-05-05 15:55:19,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:19,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-05-05 15:55:19,257 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302119257"}]},"ts":"1683302119257"} 2023-05-05 15:55:19,257 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-05-05 15:55:19,260 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-05-05 15:55:19,260 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=96da58919e3f0f706f77338d9b98be4c, UNASSIGN}] 2023-05-05 15:55:19,261 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=96da58919e3f0f706f77338d9b98be4c, UNASSIGN 2023-05-05 15:55:19,262 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=96da58919e3f0f706f77338d9b98be4c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:19,262 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1683302119262"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302119262"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302119262"}]},"ts":"1683302119262"} 2023-05-05 15:55:19,263 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure 96da58919e3f0f706f77338d9b98be4c, server=jenkins-hbase4.apache.org,43023,1683302022922}] 2023-05-05 15:55:19,357 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-05-05 15:55:19,415 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 96da58919e3f0f706f77338d9b98be4c 2023-05-05 15:55:19,415 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 96da58919e3f0f706f77338d9b98be4c, disabling compactions & flushes 2023-05-05 15:55:19,415 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. 2023-05-05 15:55:19,415 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. 2023-05-05 15:55:19,415 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. after waiting 0 ms 2023-05-05 15:55:19,415 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. 2023-05-05 15:55:19,419 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-05 15:55:19,420 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:55:19,420 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c. 2023-05-05 15:55:19,420 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 96da58919e3f0f706f77338d9b98be4c: 2023-05-05 15:55:19,421 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 96da58919e3f0f706f77338d9b98be4c 2023-05-05 15:55:19,422 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=96da58919e3f0f706f77338d9b98be4c, regionState=CLOSED 2023-05-05 15:55:19,422 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1683302119422"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302119422"}]},"ts":"1683302119422"} 2023-05-05 15:55:19,424 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-05-05 15:55:19,424 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure 96da58919e3f0f706f77338d9b98be4c, server=jenkins-hbase4.apache.org,43023,1683302022922 in 160 msec 2023-05-05 15:55:19,425 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-05-05 15:55:19,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=96da58919e3f0f706f77338d9b98be4c, UNASSIGN in 164 msec 2023-05-05 15:55:19,426 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302119426"}]},"ts":"1683302119426"} 2023-05-05 15:55:19,427 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-05-05 15:55:19,428 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-05-05 15:55:19,430 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 175 msec 2023-05-05 15:55:19,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-05-05 15:55:19,559 INFO [Listener at localhost/35117] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-05-05 15:55:19,559 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testInvalidSplitKey 2023-05-05 15:55:19,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:19,562 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:19,562 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:19,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-05-05 15:55:19,565 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c 2023-05-05 15:55:19,567 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/recovered.edits] 2023-05-05 15:55:19,570 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/24ab9032678f47018b3237626408a42b to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/24ab9032678f47018b3237626408a42b 2023-05-05 15:55:19,572 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/8903832a1daa48b1980047d77680178c to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/8903832a1daa48b1980047d77680178c 2023-05-05 15:55:19,573 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/950ac5a1b2ba426cbfa922fc922b3bba to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/950ac5a1b2ba426cbfa922fc922b3bba 2023-05-05 15:55:19,574 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/99e59478f71a4ea884924bdf39dfdbe0 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/99e59478f71a4ea884924bdf39dfdbe0 2023-05-05 15:55:19,575 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/a162e14ba5ff4b3fa56d5392e8070946 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/a162e14ba5ff4b3fa56d5392e8070946 2023-05-05 15:55:19,576 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/cf4529a05f584439911b641412e926b6 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/cf4529a05f584439911b641412e926b6 2023-05-05 15:55:19,577 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/f2521f29013344a4a467a518b217f301 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf1/f2521f29013344a4a467a518b217f301 2023-05-05 15:55:19,579 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/0147a67221af41ecb3e465723c40ea1a to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/0147a67221af41ecb3e465723c40ea1a 2023-05-05 15:55:19,580 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/4b7dc063029145a480512fc5dfcf3fa4 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/4b7dc063029145a480512fc5dfcf3fa4 2023-05-05 15:55:19,582 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/4f08ef299f5a4157b94dc10f9bea62c4 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/4f08ef299f5a4157b94dc10f9bea62c4 2023-05-05 15:55:19,582 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/52714d52ed3d41a7be4312b9b813bbd6 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/52714d52ed3d41a7be4312b9b813bbd6 2023-05-05 15:55:19,583 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/80ac8790a8154ccdb60a432b05abcd28 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/80ac8790a8154ccdb60a432b05abcd28 2023-05-05 15:55:19,584 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/c73fe31235fb404f8ae35a1fe512a8d6 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/c73fe31235fb404f8ae35a1fe512a8d6 2023-05-05 15:55:19,585 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/cdd7c4a5b9c84f7f8c0c687c1a98051e to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/cf2/cdd7c4a5b9c84f7f8c0c687c1a98051e 2023-05-05 15:55:19,588 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/recovered.edits/85.seqid to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c/recovered.edits/85.seqid 2023-05-05 15:55:19,588 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testInvalidSplitKey/96da58919e3f0f706f77338d9b98be4c 2023-05-05 15:55:19,588 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-05-05 15:55:19,590 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:19,591 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-05-05 15:55:19,595 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-05-05 15:55:19,596 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:19,596 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-05-05 15:55:19,596 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302119596"}]},"ts":"9223372036854775807"} 2023-05-05 15:55:19,597 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-05-05 15:55:19,597 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 96da58919e3f0f706f77338d9b98be4c, NAME => 'testInvalidSplitKey,,1683302117568.96da58919e3f0f706f77338d9b98be4c.', STARTKEY => '', ENDKEY => ''}] 2023-05-05 15:55:19,597 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-05-05 15:55:19,597 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683302119597"}]},"ts":"9223372036854775807"} 2023-05-05 15:55:19,598 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-05-05 15:55:19,600 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:19,602 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 42 msec 2023-05-05 15:55:19,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-05-05 15:55:19,664 INFO [Listener at localhost/35117] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-05-05 15:55:19,681 INFO [Listener at localhost/35117] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=368 (was 344) - Thread LEAK? -, OpenFileDescriptor=742 (was 653) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=186 (was 167) - SystemLoadAverage LEAK? -, ProcessCount=179 (was 179), AvailableMemoryMB=5122 (was 5183) 2023-05-05 15:55:19,697 INFO [Listener at localhost/35117] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=368, OpenFileDescriptor=742, MaxFileDescriptor=60000, SystemLoadAverage=186, ProcessCount=179, AvailableMemoryMB=5122 2023-05-05 15:55:19,697 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-05 15:55:19,699 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-05 15:55:19,699 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-05 15:55:19,701 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-05 15:55:19,702 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-05-05 15:55:19,702 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-05 15:55:19,704 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:19,704 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907 empty. 2023-05-05 15:55:19,705 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:19,705 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-05-05 15:55:19,713 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-05-05 15:55:19,714 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => 3faa7396b5050e4a3b1d1489c3bb7907, NAME => 'testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp 2023-05-05 15:55:19,721 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:55:19,721 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing 3faa7396b5050e4a3b1d1489c3bb7907, disabling compactions & flushes 2023-05-05 15:55:19,721 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. 2023-05-05 15:55:19,721 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. 2023-05-05 15:55:19,721 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. after waiting 0 ms 2023-05-05 15:55:19,721 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. 2023-05-05 15:55:19,721 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. 2023-05-05 15:55:19,721 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for 3faa7396b5050e4a3b1d1489c3bb7907: 2023-05-05 15:55:19,723 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-05-05 15:55:19,724 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683302119723"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302119723"}]},"ts":"1683302119723"} 2023-05-05 15:55:19,725 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-05 15:55:19,725 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-05 15:55:19,725 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302119725"}]},"ts":"1683302119725"} 2023-05-05 15:55:19,726 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-05-05 15:55:19,730 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-05 15:55:19,731 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-05 15:55:19,731 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-05 15:55:19,731 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-05 15:55:19,731 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-05 15:55:19,731 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907, ASSIGN}] 2023-05-05 15:55:19,732 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907, ASSIGN 2023-05-05 15:55:19,733 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43023,1683302022922; forceNewPlan=false, retain=false 2023-05-05 15:55:19,883 INFO [jenkins-hbase4:44213] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-05 15:55:19,884 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=3faa7396b5050e4a3b1d1489c3bb7907, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:19,884 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683302119884"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302119884"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302119884"}]},"ts":"1683302119884"} 2023-05-05 15:55:19,886 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 3faa7396b5050e4a3b1d1489c3bb7907, server=jenkins-hbase4.apache.org,43023,1683302022922}] 2023-05-05 15:55:20,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. 2023-05-05 15:55:20,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3faa7396b5050e4a3b1d1489c3bb7907, NAME => 'testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907.', STARTKEY => '', ENDKEY => ''} 2023-05-05 15:55:20,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:55:20,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:20,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:55:20,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:20,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:20,042 INFO [StoreOpener-3faa7396b5050e4a3b1d1489c3bb7907-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:20,043 DEBUG [StoreOpener-3faa7396b5050e4a3b1d1489c3bb7907-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1 2023-05-05 15:55:20,043 DEBUG [StoreOpener-3faa7396b5050e4a3b1d1489c3bb7907-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1 2023-05-05 15:55:20,044 INFO [StoreOpener-3faa7396b5050e4a3b1d1489c3bb7907-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3faa7396b5050e4a3b1d1489c3bb7907 columnFamilyName cf1 2023-05-05 15:55:20,044 INFO [StoreOpener-3faa7396b5050e4a3b1d1489c3bb7907-1] regionserver.HStore(310): Store=3faa7396b5050e4a3b1d1489c3bb7907/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:55:20,044 INFO [StoreOpener-3faa7396b5050e4a3b1d1489c3bb7907-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:20,045 DEBUG [StoreOpener-3faa7396b5050e4a3b1d1489c3bb7907-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2 2023-05-05 15:55:20,045 DEBUG [StoreOpener-3faa7396b5050e4a3b1d1489c3bb7907-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2 2023-05-05 15:55:20,045 INFO [StoreOpener-3faa7396b5050e4a3b1d1489c3bb7907-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3faa7396b5050e4a3b1d1489c3bb7907 columnFamilyName cf2 2023-05-05 15:55:20,046 INFO [StoreOpener-3faa7396b5050e4a3b1d1489c3bb7907-1] regionserver.HStore(310): Store=3faa7396b5050e4a3b1d1489c3bb7907/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:55:20,046 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:20,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:20,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-05 15:55:20,049 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:20,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-05 15:55:20,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3faa7396b5050e4a3b1d1489c3bb7907; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61661993, jitterRate=-0.08116470277309418}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-05 15:55:20,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3faa7396b5050e4a3b1d1489c3bb7907: 2023-05-05 15:55:20,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907., pid=111, masterSystemTime=1683302120037 2023-05-05 15:55:20,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. 2023-05-05 15:55:20,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. 2023-05-05 15:55:20,053 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=3faa7396b5050e4a3b1d1489c3bb7907, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,053 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683302120053"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302120053"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302120053"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302120053"}]},"ts":"1683302120053"} 2023-05-05 15:55:20,055 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-05-05 15:55:20,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 3faa7396b5050e4a3b1d1489c3bb7907, server=jenkins-hbase4.apache.org,43023,1683302022922 in 168 msec 2023-05-05 15:55:20,057 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-05-05 15:55:20,057 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907, ASSIGN in 325 msec 2023-05-05 15:55:20,058 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-05 15:55:20,058 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302120058"}]},"ts":"1683302120058"} 2023-05-05 15:55:20,059 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-05-05 15:55:20,061 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-05-05 15:55:20,062 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 361 msec 2023-05-05 15:55:20,208 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:55:20,209 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-05-05 15:55:20,209 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-05-05 15:55:20,209 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:20,209 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-05-05 15:55:20,209 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-05-05 15:55:20,210 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,210 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,211 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,211 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,211 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,211 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,211 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,211 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:55:20,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,212 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,212 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,212 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:20,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:20,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:20,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,212 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-05 15:55:20,212 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-05 15:55:20,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,212 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-05 15:55:20,213 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-05 15:55:20,213 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-05 15:55:20,213 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-05 15:55:20,213 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-05 15:55:20,213 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-05 15:55:20,213 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-05 15:55:20,213 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:20,213 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:20,213 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:20,213 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:55:20,213 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:55:20,213 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-05 15:55:20,213 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:55:20,213 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-05 15:55:20,213 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:55:20,213 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:55:20,213 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-05 15:55:20,213 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. 2023-05-05 15:55:20,213 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. started... 2023-05-05 15:55:20,213 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-05 15:55:20,214 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3faa7396b5050e4a3b1d1489c3bb7907 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-05 15:55:20,216 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,217 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,217 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,217 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:20,217 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,217 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:20,223 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf1/3f7a051f8fb3401a98e684e3f3c87774 2023-05-05 15:55:20,234 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf2/4253289dd83c47bc97fc4d2a3d2ecf82 2023-05-05 15:55:20,238 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf1/3f7a051f8fb3401a98e684e3f3c87774 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/3f7a051f8fb3401a98e684e3f3c87774 2023-05-05 15:55:20,241 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/3f7a051f8fb3401a98e684e3f3c87774, entries=2, sequenceid=6, filesize=4.8 K 2023-05-05 15:55:20,242 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf2/4253289dd83c47bc97fc4d2a3d2ecf82 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/4253289dd83c47bc97fc4d2a3d2ecf82 2023-05-05 15:55:20,245 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/4253289dd83c47bc97fc4d2a3d2ecf82, entries=2, sequenceid=6, filesize=4.8 K 2023-05-05 15:55:20,246 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 3faa7396b5050e4a3b1d1489c3bb7907 in 32ms, sequenceid=6, compaction requested=false 2023-05-05 15:55:20,246 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-05-05 15:55:20,246 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3faa7396b5050e4a3b1d1489c3bb7907: 2023-05-05 15:55:20,246 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. 2023-05-05 15:55:20,246 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:55:20,246 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:55:20,246 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,246 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-05 15:55:20,247 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-05 15:55:20,248 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,248 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:20,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:20,249 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,249 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:20,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:20,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:20,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:20,251 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-05-05 15:55:20,251 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ba9811[Count = 0] remaining members to acquire global barrier 2023-05-05 15:55:20,251 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-05-05 15:55:20,251 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,252 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,252 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,252 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:55:20,252 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:55:20,252 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-05 15:55:20,252 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,252 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:55:20,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,252 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-05 15:55:20,253 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:55:20,253 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,253 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:55:20,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,253 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:55:20,253 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-05 15:55:20,253 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:55:20,257 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-05 15:55:20,257 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:20,257 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-05 15:55:20,258 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-05 15:55:20,258 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:20,258 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-05 15:55:20,259 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-05 15:55:20,259 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:20,259 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,259 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-05 15:55:20,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:20,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:20,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:20,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:20,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:20,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,43023,1683302022922': 2023-05-05 15:55:20,262 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-05 15:55:20,262 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-05-05 15:55:20,262 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:55:20,262 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:55:20,262 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-05-05 15:55:20,262 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:55:20,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:20,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:20,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,264 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:20,265 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,265 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,265 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:20,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,266 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:20,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,267 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,268 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,268 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,271 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,271 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,271 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,271 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,271 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,271 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,271 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,271 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,271 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,271 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-05-05 15:55:20,272 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:55:20,271 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:20,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,272 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,272 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-05 15:55:20,272 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,272 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:55:20,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,272 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,272 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,273 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,273 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,273 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,273 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,372 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:55:20,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:55:20,384 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:55:20,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-05 15:55:20,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-05-05 15:55:20,386 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-05-05 15:55:20,386 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:20,386 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-05-05 15:55:20,386 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-05-05 15:55:20,386 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,386 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,388 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,388 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,388 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,388 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,388 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,388 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:55:20,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,389 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,389 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,389 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:20,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:20,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:20,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,389 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-05 15:55:20,389 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-05 15:55:20,389 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-05 15:55:20,390 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-05 15:55:20,390 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-05 15:55:20,390 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-05 15:55:20,390 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-05 15:55:20,390 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:20,390 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-05 15:55:20,390 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-05 15:55:20,390 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-05 15:55:20,390 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-05 15:55:20,390 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:20,390 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:55:20,390 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:55:20,390 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-05 15:55:20,390 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-05 15:55:20,390 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:20,390 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-05 15:55:20,390 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:55:20,390 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-05 15:55:20,390 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:55:20,390 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:55:20,390 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. 2023-05-05 15:55:20,390 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-05 15:55:20,390 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. started... 2023-05-05 15:55:20,391 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3faa7396b5050e4a3b1d1489c3bb7907 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:55:20,391 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,392 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,393 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,393 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:20,393 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,393 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:20,400 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf1/e7839c6630034b459434821a979cdba4 2023-05-05 15:55:20,410 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf2/0660dfdc5d6344eaadc077311ca6a079 2023-05-05 15:55:20,413 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf1/e7839c6630034b459434821a979cdba4 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/e7839c6630034b459434821a979cdba4 2023-05-05 15:55:20,417 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/e7839c6630034b459434821a979cdba4, entries=10, sequenceid=19, filesize=5.0 K 2023-05-05 15:55:20,417 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf2/0660dfdc5d6344eaadc077311ca6a079 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/0660dfdc5d6344eaadc077311ca6a079 2023-05-05 15:55:20,420 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/0660dfdc5d6344eaadc077311ca6a079, entries=10, sequenceid=19, filesize=5.0 K 2023-05-05 15:55:20,421 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3faa7396b5050e4a3b1d1489c3bb7907 in 30ms, sequenceid=19, compaction requested=false 2023-05-05 15:55:20,421 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3faa7396b5050e4a3b1d1489c3bb7907: 2023-05-05 15:55:20,421 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. 2023-05-05 15:55:20,421 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:55:20,421 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:55:20,421 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,421 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-05 15:55:20,421 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-05 15:55:20,430 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,430 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:20,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:20,431 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,431 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:20,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:20,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:20,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:20,433 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-05-05 15:55:20,433 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1ea1779[Count = 0] remaining members to acquire global barrier 2023-05-05 15:55:20,433 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-05-05 15:55:20,433 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,434 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,434 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,434 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,435 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:55:20,435 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-05 15:55:20,435 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,435 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:55:20,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,435 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:55:20,435 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:55:20,435 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-05 15:55:20,435 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:55:20,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,435 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:55:20,435 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-05 15:55:20,435 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:55:20,436 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-05 15:55:20,436 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-05 15:55:20,436 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:20,436 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:20,437 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-05 15:55:20,437 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-05 15:55:20,437 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-05 15:55:20,437 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:20,437 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,437 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-05 15:55:20,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:20,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:20,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:20,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:20,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:20,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,43023,1683302022922': 2023-05-05 15:55:20,440 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-05 15:55:20,440 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-05-05 15:55:20,440 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:55:20,440 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:55:20,440 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-05-05 15:55:20,440 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:55:20,443 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,443 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,443 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,443 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,443 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,443 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,443 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,443 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:20,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:20,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,443 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:20,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,444 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:20,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,446 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:20,446 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,446 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,448 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,448 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,448 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,448 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,449 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,449 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-05-05 15:55:20,449 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:55:20,449 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,449 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:20,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,449 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,449 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,449 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,449 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,449 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,449 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,449 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-05 15:55:20,449 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,449 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,450 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,449 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:55:20,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,450 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,450 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,550 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:55:20,551 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:55:20,561 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:55:20,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-05 15:55:20,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-05-05 15:55:20,563 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-05-05 15:55:20,563 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:20,563 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-05-05 15:55:20,563 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-05-05 15:55:20,564 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,564 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,565 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,565 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,565 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,565 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,566 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,566 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:55:20,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,566 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,566 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,566 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:20,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:20,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:20,567 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-05 15:55:20,566 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-05 15:55:20,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,567 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-05 15:55:20,567 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-05 15:55:20,567 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-05 15:55:20,567 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-05 15:55:20,567 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-05 15:55:20,567 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-05 15:55:20,567 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-05 15:55:20,567 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-05 15:55:20,567 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:20,567 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-05 15:55:20,567 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:55:20,567 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-05 15:55:20,567 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:20,567 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:20,567 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:55:20,567 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:55:20,567 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-05 15:55:20,568 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-05 15:55:20,567 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-05 15:55:20,568 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-05 15:55:20,568 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:55:20,568 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:55:20,568 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. 2023-05-05 15:55:20,568 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. started... 2023-05-05 15:55:20,568 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3faa7396b5050e4a3b1d1489c3bb7907 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:55:20,569 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,571 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,571 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,571 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:20,572 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,572 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:20,576 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf1/61ac5fa9999147bcbf73a6d18b5649cc 2023-05-05 15:55:20,589 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf2/7eade08af5844b98b3682905af5c9544 2023-05-05 15:55:20,593 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf1/61ac5fa9999147bcbf73a6d18b5649cc as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/61ac5fa9999147bcbf73a6d18b5649cc 2023-05-05 15:55:20,597 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/61ac5fa9999147bcbf73a6d18b5649cc, entries=10, sequenceid=32, filesize=5.0 K 2023-05-05 15:55:20,598 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf2/7eade08af5844b98b3682905af5c9544 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7eade08af5844b98b3682905af5c9544 2023-05-05 15:55:20,601 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7eade08af5844b98b3682905af5c9544, entries=10, sequenceid=32, filesize=5.0 K 2023-05-05 15:55:20,602 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3faa7396b5050e4a3b1d1489c3bb7907 in 34ms, sequenceid=32, compaction requested=true 2023-05-05 15:55:20,602 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3faa7396b5050e4a3b1d1489c3bb7907: 2023-05-05 15:55:20,602 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. 2023-05-05 15:55:20,602 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:55:20,602 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:55:20,602 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,602 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-05 15:55:20,602 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-05 15:55:20,604 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,604 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:20,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:20,604 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,604 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:20,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:20,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:20,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:20,606 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-05-05 15:55:20,606 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4048dcbf[Count = 0] remaining members to acquire global barrier 2023-05-05 15:55:20,606 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-05-05 15:55:20,606 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,607 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,607 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,607 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,607 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:55:20,607 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,607 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:55:20,607 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:55:20,607 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-05 15:55:20,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,608 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:55:20,608 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-05 15:55:20,608 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:55:20,608 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-05 15:55:20,608 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:55:20,608 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:55:20,609 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,609 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-05 15:55:20,610 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:20,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:20,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:20,610 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-05 15:55:20,611 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-05 15:55:20,611 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:20,611 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-05 15:55:20,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:20,611 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-05 15:55:20,611 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:20,611 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-05 15:55:20,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:20,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:20,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,43023,1683302022922': 2023-05-05 15:55:20,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-05 15:55:20,614 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-05-05 15:55:20,614 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:55:20,614 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:55:20,614 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-05-05 15:55:20,614 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:55:20,615 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,615 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,615 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,615 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:20,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:20,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,615 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,615 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,616 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:20,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,616 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,617 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,617 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:20,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,619 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:20,619 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,619 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,624 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,624 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,624 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,624 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,624 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-05-05 15:55:20,625 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:55:20,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,624 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,624 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:20,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,625 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-05 15:55:20,624 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,624 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,625 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:55:20,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,625 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,625 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,626 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,626 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,626 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,626 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,626 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,626 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,626 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,626 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,725 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:55:20,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:55:20,737 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:55:20,738 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-05 15:55:20,738 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-05-05 15:55:20,738 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-05-05 15:55:20,738 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:20,738 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-05-05 15:55:20,738 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-05-05 15:55:20,739 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,739 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,741 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,741 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,741 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,741 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,741 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,741 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:55:20,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,742 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,742 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,742 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:20,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:20,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:20,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,742 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-05 15:55:20,742 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-05 15:55:20,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,743 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-05 15:55:20,743 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-05 15:55:20,743 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-05 15:55:20,743 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-05 15:55:20,743 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-05 15:55:20,743 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-05 15:55:20,743 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-05 15:55:20,743 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:20,743 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-05 15:55:20,743 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:55:20,743 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-05 15:55:20,743 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:20,743 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-05 15:55:20,743 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:55:20,743 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-05 15:55:20,743 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-05 15:55:20,743 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-05 15:55:20,743 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:20,743 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-05 15:55:20,743 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:55:20,744 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:55:20,744 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:55:20,744 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. 2023-05-05 15:55:20,744 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. started... 2023-05-05 15:55:20,744 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3faa7396b5050e4a3b1d1489c3bb7907 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:55:20,745 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,746 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,746 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,746 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:20,746 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,746 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:20,752 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf1/08fadfa3d55d45f584fc1bfc10edd030 2023-05-05 15:55:20,766 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf2/b13ef11746c94c8b87194ebbc346f8be 2023-05-05 15:55:20,770 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf1/08fadfa3d55d45f584fc1bfc10edd030 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/08fadfa3d55d45f584fc1bfc10edd030 2023-05-05 15:55:20,774 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/08fadfa3d55d45f584fc1bfc10edd030, entries=10, sequenceid=45, filesize=5.0 K 2023-05-05 15:55:20,774 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf2/b13ef11746c94c8b87194ebbc346f8be as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/b13ef11746c94c8b87194ebbc346f8be 2023-05-05 15:55:20,778 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/b13ef11746c94c8b87194ebbc346f8be, entries=10, sequenceid=45, filesize=5.0 K 2023-05-05 15:55:20,778 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3faa7396b5050e4a3b1d1489c3bb7907 in 34ms, sequenceid=45, compaction requested=true 2023-05-05 15:55:20,778 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3faa7396b5050e4a3b1d1489c3bb7907: 2023-05-05 15:55:20,778 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. 2023-05-05 15:55:20,778 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:55:20,779 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:55:20,779 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,779 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-05 15:55:20,779 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-05 15:55:20,780 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,780 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:20,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:20,780 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,780 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:20,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:20,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:20,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:20,782 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-05-05 15:55:20,782 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35a2d26a[Count = 0] remaining members to acquire global barrier 2023-05-05 15:55:20,782 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-05-05 15:55:20,782 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,785 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,785 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,785 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,785 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:55:20,785 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-05 15:55:20,785 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:55:20,785 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-05 15:55:20,785 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,786 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:55:20,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,785 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:55:20,786 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:55:20,786 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-05 15:55:20,785 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:55:20,786 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:55:20,787 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,787 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-05 15:55:20,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,787 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-05 15:55:20,787 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:20,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:20,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:20,787 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:20,787 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-05 15:55:20,787 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-05 15:55:20,787 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:20,787 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-05 15:55:20,787 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-05 15:55:20,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:20,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:20,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:20,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,43023,1683302022922': 2023-05-05 15:55:20,790 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-05 15:55:20,790 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-05-05 15:55:20,790 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:55:20,790 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:55:20,790 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-05-05 15:55:20,790 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:55:20,792 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,792 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,792 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,792 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,792 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,792 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,792 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,792 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:20,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,793 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:20,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:20,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,793 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:20,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,795 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,795 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,795 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:20,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,798 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,798 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,798 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,798 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:20,798 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,798 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,798 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,798 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-05-05 15:55:20,798 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:55:20,798 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,798 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,798 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,798 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,799 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-05 15:55:20,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,799 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,799 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:55:20,799 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,799 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,799 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,799 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,799 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,800 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,899 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:55:20,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:55:20,911 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:55:20,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-05 15:55:20,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-05-05 15:55:20,913 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-05-05 15:55:20,913 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:20,913 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-05-05 15:55:20,913 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-05-05 15:55:20,913 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,913 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,920 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,920 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,920 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,920 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,921 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,921 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:55:20,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,921 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,921 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,922 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:20,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:20,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,922 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-05 15:55:20,922 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-05 15:55:20,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:20,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,922 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-05 15:55:20,922 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-05 15:55:20,922 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-05 15:55:20,922 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-05 15:55:20,922 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-05 15:55:20,922 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-05 15:55:20,922 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-05 15:55:20,922 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:20,922 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-05 15:55:20,922 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:20,922 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:55:20,922 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-05 15:55:20,922 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:55:20,923 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-05 15:55:20,923 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:55:20,923 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:55:20,923 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. 2023-05-05 15:55:20,923 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-05 15:55:20,923 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:20,923 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. started... 2023-05-05 15:55:20,923 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:55:20,923 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-05 15:55:20,923 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-05 15:55:20,923 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-05 15:55:20,923 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3faa7396b5050e4a3b1d1489c3bb7907 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:55:20,925 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,925 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,925 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:20,925 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,926 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,926 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:20,932 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf1/9df8d31606d24454a3f7dbc18cab3b82 2023-05-05 15:55:20,942 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf2/d5cc6bdc70b043298d32ca269e42816a 2023-05-05 15:55:20,946 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf1/9df8d31606d24454a3f7dbc18cab3b82 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9df8d31606d24454a3f7dbc18cab3b82 2023-05-05 15:55:20,949 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9df8d31606d24454a3f7dbc18cab3b82, entries=10, sequenceid=58, filesize=5.0 K 2023-05-05 15:55:20,950 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf2/d5cc6bdc70b043298d32ca269e42816a as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/d5cc6bdc70b043298d32ca269e42816a 2023-05-05 15:55:20,953 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/d5cc6bdc70b043298d32ca269e42816a, entries=10, sequenceid=58, filesize=5.0 K 2023-05-05 15:55:20,954 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3faa7396b5050e4a3b1d1489c3bb7907 in 31ms, sequenceid=58, compaction requested=true 2023-05-05 15:55:20,954 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3faa7396b5050e4a3b1d1489c3bb7907: 2023-05-05 15:55:20,954 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. 2023-05-05 15:55:20,954 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:55:20,954 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:55:20,954 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,954 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-05 15:55:20,954 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-05 15:55:20,959 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,959 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:20,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:20,960 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,960 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:20,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:20,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:20,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:20,961 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-05-05 15:55:20,961 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@678aaedf[Count = 0] remaining members to acquire global barrier 2023-05-05 15:55:20,961 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-05-05 15:55:20,962 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,963 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,963 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,963 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,963 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:55:20,963 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-05 15:55:20,963 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,963 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:55:20,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,963 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:55:20,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,963 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:55:20,963 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-05 15:55:20,963 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:55:20,963 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:55:20,964 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-05 15:55:20,964 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:55:20,965 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,965 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-05 15:55:20,965 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:20,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:20,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:20,965 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-05 15:55:20,968 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-05 15:55:20,969 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:20,969 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-05 15:55:20,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:20,969 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-05 15:55:20,969 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:20,969 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-05 15:55:20,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:20,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:20,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,43023,1683302022922': 2023-05-05 15:55:20,973 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-05 15:55:20,973 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-05-05 15:55:20,973 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:55:20,973 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:55:20,973 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-05-05 15:55:20,973 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:55:20,974 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,974 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,974 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,974 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,974 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,974 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,974 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:20,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,975 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:20,975 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:20,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,975 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:20,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,977 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:20,977 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:20,978 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,980 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,981 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,981 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,981 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,981 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,981 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:20,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,981 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-05-05 15:55:20,981 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:55:20,981 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,981 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,981 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,981 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,981 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-05 15:55:20,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,981 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:20,981 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:55:20,981 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,982 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:20,982 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:20,982 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,982 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:20,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:20,984 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:20,984 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:20,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:20,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:20,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:20,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:21,082 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:55:21,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:55:21,094 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:55:21,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-05 15:55:21,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-05-05 15:55:21,095 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-05-05 15:55:21,095 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:21,095 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-05-05 15:55:21,095 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-05-05 15:55:21,096 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,096 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,098 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:21,098 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:21,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:21,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:21,098 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:21,098 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:21,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:21,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:21,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:21,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:21,099 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:21,099 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:55:21,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,099 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,099 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,099 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:21,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:21,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:21,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,100 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-05 15:55:21,100 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-05 15:55:21,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,100 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-05 15:55:21,100 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-05 15:55:21,100 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-05 15:55:21,100 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-05 15:55:21,100 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-05 15:55:21,100 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-05 15:55:21,100 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-05 15:55:21,100 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-05 15:55:21,100 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:21,100 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:21,100 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-05 15:55:21,100 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:55:21,100 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-05 15:55:21,100 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:55:21,100 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-05 15:55:21,100 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-05 15:55:21,101 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:21,101 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-05 15:55:21,101 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-05 15:55:21,101 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:55:21,101 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:55:21,101 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. 2023-05-05 15:55:21,101 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-05 15:55:21,101 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. started... 2023-05-05 15:55:21,101 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3faa7396b5050e4a3b1d1489c3bb7907 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:55:21,103 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,104 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,104 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:21,104 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,104 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,104 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:21,109 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf1/6ea6afe5d7a04c7ebabc24c86506484b 2023-05-05 15:55:21,119 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf2/446e009084fb4a7e91db420c46a5a100 2023-05-05 15:55:21,123 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf1/6ea6afe5d7a04c7ebabc24c86506484b as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/6ea6afe5d7a04c7ebabc24c86506484b 2023-05-05 15:55:21,126 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/6ea6afe5d7a04c7ebabc24c86506484b, entries=10, sequenceid=71, filesize=5.0 K 2023-05-05 15:55:21,127 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf2/446e009084fb4a7e91db420c46a5a100 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/446e009084fb4a7e91db420c46a5a100 2023-05-05 15:55:21,130 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/446e009084fb4a7e91db420c46a5a100, entries=10, sequenceid=71, filesize=5.0 K 2023-05-05 15:55:21,130 INFO [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3faa7396b5050e4a3b1d1489c3bb7907 in 29ms, sequenceid=71, compaction requested=true 2023-05-05 15:55:21,131 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3faa7396b5050e4a3b1d1489c3bb7907: 2023-05-05 15:55:21,131 DEBUG [rs(jenkins-hbase4.apache.org,43023,1683302022922)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. 2023-05-05 15:55:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:55:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:55:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-05 15:55:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-05 15:55:21,133 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:21,133 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:21,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:21,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:21,133 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,133 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:21,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:21,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:21,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:21,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:21,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:21,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:21,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:21,135 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-05-05 15:55:21,135 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@368a3c88[Count = 0] remaining members to acquire global barrier 2023-05-05 15:55:21,135 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-05-05 15:55:21,135 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,137 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,137 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,137 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,137 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:21,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,138 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:55:21,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,138 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-05 15:55:21,138 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:55:21,138 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-05 15:55:21,138 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:55:21,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,138 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:55:21,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,138 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:55:21,138 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-05 15:55:21,138 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-05 15:55:21,138 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:55:21,139 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-05 15:55:21,139 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-05 15:55:21,139 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:21,139 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:21,139 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:21,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:21,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:21,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:21,139 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-05 15:55:21,139 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-05 15:55:21,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:21,140 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-05 15:55:21,140 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:21,140 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-05 15:55:21,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:21,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:21,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:21,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:21,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:21,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:21,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:21,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:21,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:21,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:21,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,43023,1683302022922': 2023-05-05 15:55:21,143 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-05 15:55:21,143 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-05-05 15:55:21,143 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:55:21,143 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:55:21,143 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-05-05 15:55:21,143 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:55:21,144 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,144 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,144 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,144 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:21,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:21,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:21,144 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:21,144 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,145 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:21,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,145 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:21,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:21,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:21,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:21,145 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:21,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:21,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:21,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:21,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:21,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:21,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,145 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:21,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:21,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:21,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:21,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:21,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:21,147 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:21,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:21,147 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:21,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-05 15:55:21,147 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:21,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:21,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:21,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:21,154 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:21,154 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,154 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:21,154 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:21,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:21,154 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-05-05 15:55:21,154 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:55:21,154 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:21,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:21,154 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:21,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:21,154 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:21,154 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:21,154 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:21,154 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-05 15:55:21,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:21,154 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:21,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:21,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:21,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:21,154 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:21,154 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:21,154 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:55:21,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:21,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:21,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:21,155 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:21,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:21,155 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,155 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:21,155 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:21,155 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,155 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-05 15:55:21,255 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:55:21,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:55:21,513 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-05-05 15:55:21,514 DEBUG [Listener at localhost/35117] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:21,515 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:21,516 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:21,516 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:21,516 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:21,516 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:55:21,517 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:21,517 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:55:21,517 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:55:21,765 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:21,765 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:55:21,765 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:55:21,765 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:55:21,765 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-05-05 15:55:21,766 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:55:21,766 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:55:21,766 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:55:21,766 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:55:21,766 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:55:21,768 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-05-05 15:55:21,768 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:55:21,768 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:55:21,768 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:55:21,769 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:55:21,769 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:21,769 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:55:21,769 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:21,769 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:21,769 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:21,770 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:55:21,770 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:21,770 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:21,770 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:21,770 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:55:21,771 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:55:21,771 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:21,771 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:21,771 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:21,771 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=cfa9eb043f55be3f9ab828cd40416e7f, daughterA=3296930d4aa28614ec2511a0a3f26d3c, daughterB=c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:55:21,772 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:21,772 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:21,772 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:21,772 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=df08d89907b6ac6a28b72ec117993ac2, daughterA=6cb2bd681dce908bd0c281b2c71d45e8, daughterB=2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:55:21,773 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:21,773 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:21,773 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:21,773 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:21,774 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42799,1683302022977, table=testRollbackForSplitTableRegionWithReplica, region=47d3ff38207fecb2cbad8f4f5aab825b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef3937c5a4e99c77bfe5f10af1ca890, daughterA=b385b26868fcaaccb79e222957786618, daughterB=a13170d1b393d49a10002f92ce6dcd48 2023-05-05 15:55:21,774 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:21,774 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:21,774 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:21,774 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:21,774 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:21,774 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-05 15:55:21,775 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:21,775 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:55:21,775 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:21,775 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:21,775 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:21,775 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-05-05 15:55:21,776 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:55:21,776 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:55:21,777 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302151776 2023-05-05 15:55:21,778 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:21,778 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:55:21,780 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:55:21,781 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=3faa7396b5050e4a3b1d1489c3bb7907, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:55:21,781 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:55:21,781 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302181781 2023-05-05 15:55:21,781 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302241781 2023-05-05 15:55:21,781 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-05 15:55:21,781 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:55:21,783 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:55:21,783 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:21,783 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:55:21,783 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:55:22,031 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:22,031 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:55:22,031 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:55:22,031 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:55:22,031 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:55:22,032 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:55:22,032 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:55:22,032 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:55:22,032 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:55:22,032 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:55:22,034 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-05-05 15:55:22,034 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:55:22,034 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:55:22,034 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:55:22,034 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:55:22,035 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:22,035 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:55:22,035 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:22,035 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:22,035 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:22,035 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:55:22,035 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:22,036 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:22,036 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:22,036 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:55:22,036 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:55:22,036 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:22,037 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:22,037 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:22,037 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=cfa9eb043f55be3f9ab828cd40416e7f, daughterA=3296930d4aa28614ec2511a0a3f26d3c, daughterB=c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:55:22,037 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:22,037 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:22,037 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:22,037 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=df08d89907b6ac6a28b72ec117993ac2, daughterA=6cb2bd681dce908bd0c281b2c71d45e8, daughterB=2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:55:22,037 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:22,038 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:22,038 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:22,038 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:22,038 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42799,1683302022977, table=testRollbackForSplitTableRegionWithReplica, region=47d3ff38207fecb2cbad8f4f5aab825b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef3937c5a4e99c77bfe5f10af1ca890, daughterA=b385b26868fcaaccb79e222957786618, daughterB=a13170d1b393d49a10002f92ce6dcd48 2023-05-05 15:55:22,038 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:22,038 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:22,038 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:22,039 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:22,039 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:22,039 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-05 15:55:22,039 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:22,039 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:55:22,039 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:22,039 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:22,039 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:22,040 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-05 15:55:22,040 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:55:22,041 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:55:22,041 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302152041 2023-05-05 15:55:22,042 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:55:22,042 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:22,046 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:55:22,046 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=3faa7396b5050e4a3b1d1489c3bb7907, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:55:22,046 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:55:22,046 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302182046 2023-05-05 15:55:22,046 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302242046 2023-05-05 15:55:22,046 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-05 15:55:22,046 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:55:22,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907, UNASSIGN}] 2023-05-05 15:55:22,048 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:55:22,048 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:22,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:55:22,048 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:55:22,296 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:22,296 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:55:22,296 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:55:22,297 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:55:22,297 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:55:22,297 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:55:22,297 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:55:22,297 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:55:22,297 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:55:22,298 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:55:22,299 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-05-05 15:55:22,300 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:55:22,300 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:55:22,301 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:55:22,301 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:55:22,301 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:22,301 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:55:22,302 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:22,302 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:22,302 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:22,302 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:55:22,302 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:22,303 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:22,303 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:22,303 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:55:22,303 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:55:22,304 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:22,304 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:22,304 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:22,304 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=cfa9eb043f55be3f9ab828cd40416e7f, daughterA=3296930d4aa28614ec2511a0a3f26d3c, daughterB=c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:55:22,304 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:22,305 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:22,305 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:22,305 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=df08d89907b6ac6a28b72ec117993ac2, daughterA=6cb2bd681dce908bd0c281b2c71d45e8, daughterB=2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:55:22,305 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:22,306 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:22,306 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:22,306 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:22,307 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42799,1683302022977, table=testRollbackForSplitTableRegionWithReplica, region=47d3ff38207fecb2cbad8f4f5aab825b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef3937c5a4e99c77bfe5f10af1ca890, daughterA=b385b26868fcaaccb79e222957786618, daughterB=a13170d1b393d49a10002f92ce6dcd48 2023-05-05 15:55:22,307 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:22,307 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:22,307 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:22,307 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:22,308 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:22,308 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-05 15:55:22,308 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:22,308 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:55:22,308 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:22,308 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:22,309 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:22,309 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-05-05 15:55:22,309 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:55:22,310 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:55:22,310 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302152310 2023-05-05 15:55:22,312 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:22,312 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:55:22,314 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:55:22,315 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=3faa7396b5050e4a3b1d1489c3bb7907, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:55:22,315 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:55:22,315 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302182315 2023-05-05 15:55:22,315 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302242315 2023-05-05 15:55:22,315 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-05 15:55:22,315 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907, UNASSIGN}] 2023-05-05 15:55:22,316 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:55:22,318 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907, UNASSIGN 2023-05-05 15:55:22,318 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=3faa7396b5050e4a3b1d1489c3bb7907, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:22,319 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683302122318"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302122318"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302122318"}]},"ts":"1683302122318"} 2023-05-05 15:55:22,321 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure 3faa7396b5050e4a3b1d1489c3bb7907, server=jenkins-hbase4.apache.org,43023,1683302022922}] 2023-05-05 15:55:22,473 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,473 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3faa7396b5050e4a3b1d1489c3bb7907, disabling compactions & flushes 2023-05-05 15:55:22,473 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. 2023-05-05 15:55:22,473 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. 2023-05-05 15:55:22,473 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. after waiting 0 ms 2023-05-05 15:55:22,473 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. 2023-05-05 15:55:22,473 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 3faa7396b5050e4a3b1d1489c3bb7907 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-05 15:55:22,486 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf1/9bdb81d2f49147a9a352edb000b4f39a 2023-05-05 15:55:22,497 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf2/7569bf47fa6145f38deffbf61231360f 2023-05-05 15:55:22,500 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf1/9bdb81d2f49147a9a352edb000b4f39a as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9bdb81d2f49147a9a352edb000b4f39a 2023-05-05 15:55:22,504 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9bdb81d2f49147a9a352edb000b4f39a, entries=8, sequenceid=82, filesize=5.0 K 2023-05-05 15:55:22,504 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/.tmp/cf2/7569bf47fa6145f38deffbf61231360f as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7569bf47fa6145f38deffbf61231360f 2023-05-05 15:55:22,509 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7569bf47fa6145f38deffbf61231360f, entries=8, sequenceid=82, filesize=5.0 K 2023-05-05 15:55:22,509 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 3faa7396b5050e4a3b1d1489c3bb7907 in 36ms, sequenceid=82, compaction requested=true 2023-05-05 15:55:22,513 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-05 15:55:22,513 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:55:22,514 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. 2023-05-05 15:55:22,514 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3faa7396b5050e4a3b1d1489c3bb7907: 2023-05-05 15:55:22,516 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,516 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=3faa7396b5050e4a3b1d1489c3bb7907, regionState=CLOSED 2023-05-05 15:55:22,517 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683302122516"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302122516"}]},"ts":"1683302122516"} 2023-05-05 15:55:22,519 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-05-05 15:55:22,519 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3faa7396b5050e4a3b1d1489c3bb7907, server=jenkins-hbase4.apache.org,43023,1683302022922 in 196 msec 2023-05-05 15:55:22,520 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-05-05 15:55:22,520 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907, UNASSIGN in 204 msec 2023-05-05 15:55:22,522 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:55:22,522 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:22,522 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:55:22,522 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:55:22,565 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:22,566 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:55:22,566 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:55:22,566 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:55:22,566 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:55:22,566 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:55:22,566 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:55:22,566 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:55:22,566 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:55:22,567 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:55:22,568 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-05 15:55:22,568 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-05 15:55:22,568 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-05 15:55:22,568 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:55:22,568 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:55:22,569 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:55:22,569 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:55:22,569 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:22,569 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:55:22,569 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:22,569 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:22,569 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:22,570 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:55:22,570 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:22,570 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:22,570 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:22,570 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:55:22,570 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:55:22,570 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:22,571 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:22,571 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:22,571 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=cfa9eb043f55be3f9ab828cd40416e7f, daughterA=3296930d4aa28614ec2511a0a3f26d3c, daughterB=c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:55:22,571 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:22,571 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:22,571 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:22,571 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=df08d89907b6ac6a28b72ec117993ac2, daughterA=6cb2bd681dce908bd0c281b2c71d45e8, daughterB=2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:55:22,572 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:22,572 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:22,572 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:22,572 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:22,572 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42799,1683302022977, table=testRollbackForSplitTableRegionWithReplica, region=47d3ff38207fecb2cbad8f4f5aab825b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef3937c5a4e99c77bfe5f10af1ca890, daughterA=b385b26868fcaaccb79e222957786618, daughterB=a13170d1b393d49a10002f92ce6dcd48 2023-05-05 15:55:22,572 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:22,573 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:22,573 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:22,573 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:22,573 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:22,573 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-05 15:55:22,574 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:22,574 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907, UNASSIGN 2023-05-05 15:55:22,574 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3faa7396b5050e4a3b1d1489c3bb7907, server=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:22,574 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:55:22,574 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:22,575 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:22,575 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:22,575 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:55:22,575 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3faa7396b5050e4a3b1d1489c3bb7907, server=jenkins-hbase4.apache.org,43023,1683302022922 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:55:22,575 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-05-05 15:55:22,576 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:55:22,576 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:55:22,577 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302152577 2023-05-05 15:55:22,579 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:55:22,579 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:22,581 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:55:22,581 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=3faa7396b5050e4a3b1d1489c3bb7907, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:55:22,581 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:55:22,581 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302182581 2023-05-05 15:55:22,581 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302242581 2023-05-05 15:55:22,581 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-05 15:55:22,584 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:55:22,592 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=3faa7396b5050e4a3b1d1489c3bb7907, threads=14 2023-05-05 15:55:22,593 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/0660dfdc5d6344eaadc077311ca6a079 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,593 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/4253289dd83c47bc97fc4d2a3d2ecf82 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,593 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/446e009084fb4a7e91db420c46a5a100 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,593 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7569bf47fa6145f38deffbf61231360f for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,593 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7eade08af5844b98b3682905af5c9544 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,593 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/b13ef11746c94c8b87194ebbc346f8be for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,594 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/d5cc6bdc70b043298d32ca269e42816a for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,596 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/08fadfa3d55d45f584fc1bfc10edd030 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,596 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/3f7a051f8fb3401a98e684e3f3c87774 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,596 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/6ea6afe5d7a04c7ebabc24c86506484b for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,597 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9bdb81d2f49147a9a352edb000b4f39a for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,597 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/61ac5fa9999147bcbf73a6d18b5649cc for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,600 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9df8d31606d24454a3f7dbc18cab3b82 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,600 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/e7839c6630034b459434821a979cdba4 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,753 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9bdb81d2f49147a9a352edb000b4f39a for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,760 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/0660dfdc5d6344eaadc077311ca6a079 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,766 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/446e009084fb4a7e91db420c46a5a100 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,797 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/08fadfa3d55d45f584fc1bfc10edd030 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,800 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7569bf47fa6145f38deffbf61231360f for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,803 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9df8d31606d24454a3f7dbc18cab3b82 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,803 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/4253289dd83c47bc97fc4d2a3d2ecf82 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,805 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/e7839c6630034b459434821a979cdba4 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,805 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7eade08af5844b98b3682905af5c9544 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,805 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/3f7a051f8fb3401a98e684e3f3c87774 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,806 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/6ea6afe5d7a04c7ebabc24c86506484b for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,808 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/d5cc6bdc70b043298d32ca269e42816a for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,810 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/b13ef11746c94c8b87194ebbc346f8be for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,813 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/61ac5fa9999147bcbf73a6d18b5649cc for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:22,813 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 3faa7396b5050e4a3b1d1489c3bb7907 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-05 15:55:22,841 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:55:22,841 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:22,841 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:55:22,841 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:55:22,842 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-05 15:55:23,082 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,082 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:55:23,082 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:55:23,082 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:55:23,082 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:55:23,082 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:55:23,082 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:55:23,082 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:55:23,082 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:55:23,083 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:55:23,084 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-05-05 15:55:23,084 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-05 15:55:23,084 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-05 15:55:23,085 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:55:23,085 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:55:23,085 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:55:23,085 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:55:23,085 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:23,085 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:55:23,086 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:23,086 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:23,086 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:23,086 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:55:23,086 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:23,086 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:23,086 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:23,087 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:55:23,087 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:55:23,087 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:23,087 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:23,087 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:23,088 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=cfa9eb043f55be3f9ab828cd40416e7f, daughterA=3296930d4aa28614ec2511a0a3f26d3c, daughterB=c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:55:23,088 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:23,088 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:23,088 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:23,089 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=df08d89907b6ac6a28b72ec117993ac2, daughterA=6cb2bd681dce908bd0c281b2c71d45e8, daughterB=2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:55:23,089 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:23,089 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:23,089 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:23,089 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:23,090 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42799,1683302022977, table=testRollbackForSplitTableRegionWithReplica, region=47d3ff38207fecb2cbad8f4f5aab825b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef3937c5a4e99c77bfe5f10af1ca890, daughterA=b385b26868fcaaccb79e222957786618, daughterB=a13170d1b393d49a10002f92ce6dcd48 2023-05-05 15:55:23,090 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:23,090 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:23,090 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:23,090 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:23,091 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:23,091 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-05 15:55:23,091 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,091 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907, UNASSIGN 2023-05-05 15:55:23,092 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3faa7396b5050e4a3b1d1489c3bb7907, server=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:23,092 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:55:23,092 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,092 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,092 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,092 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:55:23,092 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3faa7396b5050e4a3b1d1489c3bb7907, server=jenkins-hbase4.apache.org,43023,1683302022922 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:55:23,092 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-05-05 15:55:23,093 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:55:23,094 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:55:23,094 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302153094 2023-05-05 15:55:23,095 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,096 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:55:23,098 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:55:23,099 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=3faa7396b5050e4a3b1d1489c3bb7907, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:55:23,099 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:55:23,099 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302183099 2023-05-05 15:55:23,099 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302243099 2023-05-05 15:55:23,099 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-05 15:55:23,107 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=3faa7396b5050e4a3b1d1489c3bb7907, threads=14 2023-05-05 15:55:23,107 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/0660dfdc5d6344eaadc077311ca6a079 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,107 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/4253289dd83c47bc97fc4d2a3d2ecf82 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,108 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/446e009084fb4a7e91db420c46a5a100 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,108 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7569bf47fa6145f38deffbf61231360f for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,108 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7eade08af5844b98b3682905af5c9544 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,108 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/b13ef11746c94c8b87194ebbc346f8be for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,108 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/d5cc6bdc70b043298d32ca269e42816a for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,108 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/08fadfa3d55d45f584fc1bfc10edd030 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,109 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/3f7a051f8fb3401a98e684e3f3c87774 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,109 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/61ac5fa9999147bcbf73a6d18b5649cc for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,109 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/6ea6afe5d7a04c7ebabc24c86506484b for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,112 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9bdb81d2f49147a9a352edb000b4f39a for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,112 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9df8d31606d24454a3f7dbc18cab3b82 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,112 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/e7839c6630034b459434821a979cdba4 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,193 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/e7839c6630034b459434821a979cdba4 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,210 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/b13ef11746c94c8b87194ebbc346f8be for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,240 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/d5cc6bdc70b043298d32ca269e42816a for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,242 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7eade08af5844b98b3682905af5c9544 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,262 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/4253289dd83c47bc97fc4d2a3d2ecf82 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,263 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/6ea6afe5d7a04c7ebabc24c86506484b for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,264 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/446e009084fb4a7e91db420c46a5a100 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,276 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/0660dfdc5d6344eaadc077311ca6a079 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,280 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7569bf47fa6145f38deffbf61231360f for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,280 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9bdb81d2f49147a9a352edb000b4f39a for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,281 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/08fadfa3d55d45f584fc1bfc10edd030 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,282 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9df8d31606d24454a3f7dbc18cab3b82 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,283 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/61ac5fa9999147bcbf73a6d18b5649cc for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,283 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/3f7a051f8fb3401a98e684e3f3c87774 for region: 3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,283 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 3faa7396b5050e4a3b1d1489c3bb7907 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-05 15:55:23,315 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:55:23,319 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-05 15:55:23,321 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-05 15:55:23,321 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:55:23,321 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,321 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:55:23,321 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:55:23,349 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,349 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:55:23,349 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:55:23,349 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:55:23,350 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:55:23,350 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:55:23,350 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:55:23,350 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:55:23,350 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:55:23,350 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:55:23,352 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-05-05 15:55:23,352 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-05 15:55:23,352 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-05 15:55:23,352 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:55:23,353 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:55:23,353 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:55:23,353 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:55:23,353 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:23,354 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:55:23,354 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:23,354 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:23,354 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:23,355 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:55:23,355 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:23,355 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:23,355 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:23,356 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:55:23,356 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:55:23,356 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:23,356 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:23,356 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:23,357 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=cfa9eb043f55be3f9ab828cd40416e7f, daughterA=3296930d4aa28614ec2511a0a3f26d3c, daughterB=c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:55:23,357 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:23,357 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:23,357 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:23,357 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=df08d89907b6ac6a28b72ec117993ac2, daughterA=6cb2bd681dce908bd0c281b2c71d45e8, daughterB=2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:55:23,358 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:23,358 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:23,358 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:23,358 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:23,359 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42799,1683302022977, table=testRollbackForSplitTableRegionWithReplica, region=47d3ff38207fecb2cbad8f4f5aab825b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef3937c5a4e99c77bfe5f10af1ca890, daughterA=b385b26868fcaaccb79e222957786618, daughterB=a13170d1b393d49a10002f92ce6dcd48 2023-05-05 15:55:23,359 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:23,359 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:23,359 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:23,359 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:23,360 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:23,360 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-05 15:55:23,360 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,360 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907, UNASSIGN 2023-05-05 15:55:23,360 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3faa7396b5050e4a3b1d1489c3bb7907, server=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:23,361 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:55:23,361 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,361 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,361 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,361 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:55:23,361 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3faa7396b5050e4a3b1d1489c3bb7907, server=jenkins-hbase4.apache.org,43023,1683302022922 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:55:23,361 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-05-05 15:55:23,362 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:55:23,362 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:55:23,363 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302153363 2023-05-05 15:55:23,365 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:55:23,365 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,367 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:55:23,367 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=3faa7396b5050e4a3b1d1489c3bb7907, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:55:23,367 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:55:23,367 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302183367 2023-05-05 15:55:23,367 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302243367 2023-05-05 15:55:23,368 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-05 15:55:23,370 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:55:23,372 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:55:23,372 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:55:23,372 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:55:23,618 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,618 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:55:23,618 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:55:23,618 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:55:23,618 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:55:23,619 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:55:23,619 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:55:23,619 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:55:23,619 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:55:23,620 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:55:23,621 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-05-05 15:55:23,621 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-05 15:55:23,621 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-05 15:55:23,622 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:55:23,622 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:55:23,622 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:55:23,622 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:55:23,623 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:23,623 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:55:23,623 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:23,623 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:23,623 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:23,624 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:55:23,624 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:23,624 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:23,624 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:23,625 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:55:23,625 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:55:23,625 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:23,625 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:23,625 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:23,625 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=cfa9eb043f55be3f9ab828cd40416e7f, daughterA=3296930d4aa28614ec2511a0a3f26d3c, daughterB=c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:55:23,626 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:23,626 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:23,626 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:23,626 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=df08d89907b6ac6a28b72ec117993ac2, daughterA=6cb2bd681dce908bd0c281b2c71d45e8, daughterB=2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:55:23,626 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:23,626 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:23,627 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:23,627 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:23,627 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42799,1683302022977, table=testRollbackForSplitTableRegionWithReplica, region=47d3ff38207fecb2cbad8f4f5aab825b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef3937c5a4e99c77bfe5f10af1ca890, daughterA=b385b26868fcaaccb79e222957786618, daughterB=a13170d1b393d49a10002f92ce6dcd48 2023-05-05 15:55:23,627 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:23,628 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:23,628 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:23,628 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:23,628 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:23,629 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-05 15:55:23,629 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,629 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907, UNASSIGN 2023-05-05 15:55:23,629 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3faa7396b5050e4a3b1d1489c3bb7907, server=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:23,629 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:55:23,630 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,630 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,630 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,630 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:55:23,630 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3faa7396b5050e4a3b1d1489c3bb7907, server=jenkins-hbase4.apache.org,43023,1683302022922 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:55:23,630 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-05-05 15:55:23,631 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:55:23,631 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:55:23,632 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302153632 2023-05-05 15:55:23,633 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,633 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:55:23,636 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:55:23,636 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=3faa7396b5050e4a3b1d1489c3bb7907, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:55:23,636 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:55:23,636 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302183636 2023-05-05 15:55:23,636 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302243636 2023-05-05 15:55:23,636 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-05 15:55:23,637 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:55:23,638 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683302123638"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1683302123638"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1683302123638"}]},"ts":"1683302123638"} 2023-05-05 15:55:23,639 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683302123638"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302123638"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302123638"}]},"ts":"1683302123638"} 2023-05-05 15:55:23,639 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683302123638"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302123638"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302123638"}]},"ts":"1683302123638"} 2023-05-05 15:55:23,640 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:55:23,640 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,640 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:55:23,640 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:55:23,887 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,887 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:55:23,887 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:55:23,887 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:55:23,887 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:55:23,887 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:55:23,888 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:55:23,888 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:55:23,888 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:55:23,889 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:55:23,890 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-05-05 15:55:23,890 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-05 15:55:23,891 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-05 15:55:23,891 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:55:23,891 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:55:23,892 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:55:23,892 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:55:23,892 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:23,892 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:55:23,893 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:23,893 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:23,893 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:23,893 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:55:23,893 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:23,894 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:23,894 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:23,894 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:55:23,895 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:55:23,895 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:23,895 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:23,895 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:23,895 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=cfa9eb043f55be3f9ab828cd40416e7f, daughterA=3296930d4aa28614ec2511a0a3f26d3c, daughterB=c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:55:23,896 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:23,896 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:23,896 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:23,896 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=df08d89907b6ac6a28b72ec117993ac2, daughterA=6cb2bd681dce908bd0c281b2c71d45e8, daughterB=2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:55:23,896 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:23,897 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:23,897 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:23,897 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:23,898 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42799,1683302022977, table=testRollbackForSplitTableRegionWithReplica, region=47d3ff38207fecb2cbad8f4f5aab825b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef3937c5a4e99c77bfe5f10af1ca890, daughterA=b385b26868fcaaccb79e222957786618, daughterB=a13170d1b393d49a10002f92ce6dcd48 2023-05-05 15:55:23,898 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:23,898 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:23,898 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:23,898 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:23,899 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:23,899 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-05 15:55:23,899 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,899 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907, UNASSIGN 2023-05-05 15:55:23,899 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3faa7396b5050e4a3b1d1489c3bb7907, server=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:23,900 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:55:23,900 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,900 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,900 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,900 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:55:23,900 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3faa7396b5050e4a3b1d1489c3bb7907, server=jenkins-hbase4.apache.org,43023,1683302022922 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:55:23,900 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-05-05 15:55:23,901 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:55:23,902 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:55:23,902 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302153902 2023-05-05 15:55:23,903 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:55:23,904 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,907 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:55:23,907 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=3faa7396b5050e4a3b1d1489c3bb7907, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:55:23,907 INFO [Listener at localhost/35117] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,43023,1683302022922, table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:23,907 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=c2b9dc0d93174d9e20cc3a4a80b2d71a, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-05-05 15:55:23,907 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=64a3851b586af16bdf7346323bb046b0, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-05-05 15:55:23,907 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:55:23,907 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302183907 2023-05-05 15:55:23,907 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302243907 2023-05-05 15:55:23,908 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-05 15:55:23,908 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683302123908"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1683302123908"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1683302123908"}]},"ts":"1683302123908"} 2023-05-05 15:55:23,908 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683302123908"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302123908"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302123908"}]},"ts":"1683302123908"} 2023-05-05 15:55:23,908 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683302123908"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302123908"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302123908"}]},"ts":"1683302123908"} 2023-05-05 15:55:23,910 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:55:23,912 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:55:23,912 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:23,912 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:55:23,912 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:55:24,158 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:24,158 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:55:24,158 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:55:24,158 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:55:24,158 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:55:24,158 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:55:24,158 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:55:24,158 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:55:24,158 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:55:24,159 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:55:24,160 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-05-05 15:55:24,160 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-05 15:55:24,160 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-05 15:55:24,161 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:55:24,161 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:55:24,161 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:55:24,161 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:55:24,161 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:24,162 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:55:24,162 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:24,162 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:24,162 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:24,162 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:55:24,162 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:24,162 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:24,162 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:24,163 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:55:24,163 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:55:24,163 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:24,163 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:24,163 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:24,163 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=cfa9eb043f55be3f9ab828cd40416e7f, daughterA=3296930d4aa28614ec2511a0a3f26d3c, daughterB=c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:55:24,163 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:24,163 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:24,164 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:24,164 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=df08d89907b6ac6a28b72ec117993ac2, daughterA=6cb2bd681dce908bd0c281b2c71d45e8, daughterB=2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:55:24,164 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:24,164 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:24,164 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:24,164 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:24,164 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42799,1683302022977, table=testRollbackForSplitTableRegionWithReplica, region=47d3ff38207fecb2cbad8f4f5aab825b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef3937c5a4e99c77bfe5f10af1ca890, daughterA=b385b26868fcaaccb79e222957786618, daughterB=a13170d1b393d49a10002f92ce6dcd48 2023-05-05 15:55:24,164 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:24,165 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:24,165 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:24,165 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:24,165 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:24,165 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-05 15:55:24,165 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:24,165 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907, UNASSIGN 2023-05-05 15:55:24,165 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3faa7396b5050e4a3b1d1489c3bb7907, server=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:24,165 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:55:24,166 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:24,166 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:24,166 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:24,166 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:55:24,166 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3faa7396b5050e4a3b1d1489c3bb7907, server=jenkins-hbase4.apache.org,43023,1683302022922 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:55:24,166 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-05-05 15:55:24,166 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:55:24,167 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:55:24,167 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302154167 2023-05-05 15:55:24,169 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:24,169 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:55:24,171 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:55:24,171 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=3faa7396b5050e4a3b1d1489c3bb7907, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:55:24,171 INFO [Listener at localhost/35117] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,43023,1683302022922, table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:24,172 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=c2b9dc0d93174d9e20cc3a4a80b2d71a, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-05-05 15:55:24,172 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=64a3851b586af16bdf7346323bb046b0, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-05-05 15:55:24,172 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:55:24,172 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302184172 2023-05-05 15:55:24,172 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302244172 2023-05-05 15:55:24,172 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-05 15:55:24,172 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:55:24,174 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c2b9dc0d93174d9e20cc3a4a80b2d71a, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=64a3851b586af16bdf7346323bb046b0, ASSIGN}] 2023-05-05 15:55:24,174 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:55:24,174 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:24,174 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:55:24,174 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:55:24,422 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:24,423 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:55:24,423 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:55:24,423 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:55:24,423 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:55:24,423 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:55:24,423 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:55:24,423 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:55:24,423 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:55:24,424 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:55:24,425 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-05-05 15:55:24,425 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-05 15:55:24,425 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-05 15:55:24,425 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:55:24,425 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:55:24,426 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:55:24,426 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:55:24,426 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:24,426 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:55:24,426 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:24,426 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:24,426 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:24,426 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:55:24,427 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:24,427 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:24,427 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:24,427 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:55:24,427 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:55:24,427 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:24,427 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:24,427 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:24,428 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=cfa9eb043f55be3f9ab828cd40416e7f, daughterA=3296930d4aa28614ec2511a0a3f26d3c, daughterB=c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:55:24,428 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:24,428 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:24,428 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:24,428 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=df08d89907b6ac6a28b72ec117993ac2, daughterA=6cb2bd681dce908bd0c281b2c71d45e8, daughterB=2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:55:24,428 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:24,428 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:24,428 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:24,428 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:24,429 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42799,1683302022977, table=testRollbackForSplitTableRegionWithReplica, region=47d3ff38207fecb2cbad8f4f5aab825b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef3937c5a4e99c77bfe5f10af1ca890, daughterA=b385b26868fcaaccb79e222957786618, daughterB=a13170d1b393d49a10002f92ce6dcd48 2023-05-05 15:55:24,429 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:24,429 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:24,429 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:24,429 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:24,429 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:24,429 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-05 15:55:24,430 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:24,430 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907, UNASSIGN 2023-05-05 15:55:24,430 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3faa7396b5050e4a3b1d1489c3bb7907, server=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:24,430 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:55:24,430 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:24,430 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:24,430 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:24,430 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:55:24,430 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3faa7396b5050e4a3b1d1489c3bb7907, server=jenkins-hbase4.apache.org,43023,1683302022922 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:55:24,430 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-05-05 15:55:24,431 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:55:24,431 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:55:24,432 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302154432 2023-05-05 15:55:24,432 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:55:24,433 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:24,436 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:55:24,436 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=3faa7396b5050e4a3b1d1489c3bb7907, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:55:24,436 INFO [Listener at localhost/35117] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,43023,1683302022922, table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:24,437 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=c2b9dc0d93174d9e20cc3a4a80b2d71a, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-05-05 15:55:24,437 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=64a3851b586af16bdf7346323bb046b0, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-05-05 15:55:24,437 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:55:24,437 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302184437 2023-05-05 15:55:24,437 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302244437 2023-05-05 15:55:24,437 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-05-05 15:55:24,437 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c2b9dc0d93174d9e20cc3a4a80b2d71a, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=64a3851b586af16bdf7346323bb046b0, ASSIGN}] 2023-05-05 15:55:24,437 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:55:24,439 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=64a3851b586af16bdf7346323bb046b0, ASSIGN 2023-05-05 15:55:24,440 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=64a3851b586af16bdf7346323bb046b0, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,43023,1683302022922; forceNewPlan=false, retain=false 2023-05-05 15:55:24,440 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c2b9dc0d93174d9e20cc3a4a80b2d71a, ASSIGN 2023-05-05 15:55:24,441 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c2b9dc0d93174d9e20cc3a4a80b2d71a, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,43023,1683302022922; forceNewPlan=false, retain=false 2023-05-05 15:55:24,590 INFO [jenkins-hbase4:44213] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-05 15:55:24,591 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=c2b9dc0d93174d9e20cc3a4a80b2d71a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:24,592 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683302124591"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302124591"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302124591"}]},"ts":"1683302124591"} 2023-05-05 15:55:24,593 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=115, state=RUNNABLE; OpenRegionProcedure c2b9dc0d93174d9e20cc3a4a80b2d71a, server=jenkins-hbase4.apache.org,43023,1683302022922}] 2023-05-05 15:55:24,595 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=64a3851b586af16bdf7346323bb046b0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:24,595 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683302124595"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302124595"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302124595"}]},"ts":"1683302124595"} 2023-05-05 15:55:24,596 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=116, state=RUNNABLE; OpenRegionProcedure 64a3851b586af16bdf7346323bb046b0, server=jenkins-hbase4.apache.org,43023,1683302022922}] 2023-05-05 15:55:24,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0. 2023-05-05 15:55:24,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 64a3851b586af16bdf7346323bb046b0, NAME => 'testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.', STARTKEY => '41', ENDKEY => ''} 2023-05-05 15:55:24,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:55:24,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:24,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:55:24,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:24,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:24,749 INFO [StoreOpener-64a3851b586af16bdf7346323bb046b0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:24,750 DEBUG [StoreOpener-64a3851b586af16bdf7346323bb046b0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1 2023-05-05 15:55:24,750 DEBUG [StoreOpener-64a3851b586af16bdf7346323bb046b0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1 2023-05-05 15:55:24,750 INFO [StoreOpener-64a3851b586af16bdf7346323bb046b0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 64a3851b586af16bdf7346323bb046b0 columnFamilyName cf1 2023-05-05 15:55:24,761 DEBUG [StoreOpener-64a3851b586af16bdf7346323bb046b0-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/08fadfa3d55d45f584fc1bfc10edd030.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/08fadfa3d55d45f584fc1bfc10edd030-top 2023-05-05 15:55:24,764 DEBUG [StoreOpener-64a3851b586af16bdf7346323bb046b0-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/3f7a051f8fb3401a98e684e3f3c87774.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/3f7a051f8fb3401a98e684e3f3c87774-top 2023-05-05 15:55:24,767 DEBUG [StoreOpener-64a3851b586af16bdf7346323bb046b0-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/61ac5fa9999147bcbf73a6d18b5649cc.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/61ac5fa9999147bcbf73a6d18b5649cc-top 2023-05-05 15:55:24,770 DEBUG [StoreOpener-64a3851b586af16bdf7346323bb046b0-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/6ea6afe5d7a04c7ebabc24c86506484b.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/6ea6afe5d7a04c7ebabc24c86506484b-top 2023-05-05 15:55:24,773 DEBUG [StoreOpener-64a3851b586af16bdf7346323bb046b0-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/9bdb81d2f49147a9a352edb000b4f39a.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9bdb81d2f49147a9a352edb000b4f39a-top 2023-05-05 15:55:24,776 DEBUG [StoreOpener-64a3851b586af16bdf7346323bb046b0-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/9df8d31606d24454a3f7dbc18cab3b82.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9df8d31606d24454a3f7dbc18cab3b82-top 2023-05-05 15:55:24,778 DEBUG [StoreOpener-64a3851b586af16bdf7346323bb046b0-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/e7839c6630034b459434821a979cdba4.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/e7839c6630034b459434821a979cdba4-top 2023-05-05 15:55:24,779 INFO [StoreOpener-64a3851b586af16bdf7346323bb046b0-1] regionserver.HStore(310): Store=64a3851b586af16bdf7346323bb046b0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:55:24,779 INFO [StoreOpener-64a3851b586af16bdf7346323bb046b0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:24,779 DEBUG [StoreOpener-64a3851b586af16bdf7346323bb046b0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2 2023-05-05 15:55:24,779 DEBUG [StoreOpener-64a3851b586af16bdf7346323bb046b0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2 2023-05-05 15:55:24,780 INFO [StoreOpener-64a3851b586af16bdf7346323bb046b0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 64a3851b586af16bdf7346323bb046b0 columnFamilyName cf2 2023-05-05 15:55:24,792 DEBUG [StoreOpener-64a3851b586af16bdf7346323bb046b0-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/0660dfdc5d6344eaadc077311ca6a079.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/0660dfdc5d6344eaadc077311ca6a079-top 2023-05-05 15:55:24,795 DEBUG [StoreOpener-64a3851b586af16bdf7346323bb046b0-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/4253289dd83c47bc97fc4d2a3d2ecf82.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/4253289dd83c47bc97fc4d2a3d2ecf82-top 2023-05-05 15:55:24,798 DEBUG [StoreOpener-64a3851b586af16bdf7346323bb046b0-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/446e009084fb4a7e91db420c46a5a100.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/446e009084fb4a7e91db420c46a5a100-top 2023-05-05 15:55:24,804 DEBUG [StoreOpener-64a3851b586af16bdf7346323bb046b0-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/7569bf47fa6145f38deffbf61231360f.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7569bf47fa6145f38deffbf61231360f-top 2023-05-05 15:55:24,808 DEBUG [StoreOpener-64a3851b586af16bdf7346323bb046b0-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/7eade08af5844b98b3682905af5c9544.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7eade08af5844b98b3682905af5c9544-top 2023-05-05 15:55:24,811 DEBUG [StoreOpener-64a3851b586af16bdf7346323bb046b0-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/b13ef11746c94c8b87194ebbc346f8be.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/b13ef11746c94c8b87194ebbc346f8be-top 2023-05-05 15:55:24,815 DEBUG [StoreOpener-64a3851b586af16bdf7346323bb046b0-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/d5cc6bdc70b043298d32ca269e42816a.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/d5cc6bdc70b043298d32ca269e42816a-top 2023-05-05 15:55:24,815 INFO [StoreOpener-64a3851b586af16bdf7346323bb046b0-1] regionserver.HStore(310): Store=64a3851b586af16bdf7346323bb046b0/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:55:24,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:24,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:24,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-05 15:55:24,821 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:24,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 64a3851b586af16bdf7346323bb046b0; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69985321, jitterRate=0.04286254942417145}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-05 15:55:24,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 64a3851b586af16bdf7346323bb046b0: 2023-05-05 15:55:24,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0., pid=118, masterSystemTime=1683302124744 2023-05-05 15:55:24,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:55:24,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-05 15:55:24,826 DEBUG [RS:0;jenkins-hbase4:43023-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-05 15:55:24,826 DEBUG [RS:0;jenkins-hbase4:43023-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-05 15:55:24,826 INFO [RS:0;jenkins-hbase4:43023-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0. 2023-05-05 15:55:24,826 DEBUG [RS:0;jenkins-hbase4:43023-shortCompactions-0] regionserver.HStore(1912): 64a3851b586af16bdf7346323bb046b0/cf1 is initiating minor compaction (all files) 2023-05-05 15:55:24,826 INFO [RS:0;jenkins-hbase4:43023-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0. 2023-05-05 15:55:24,826 DEBUG [RS:0;jenkins-hbase4:43023-longCompactions-0] regionserver.HStore(1912): 64a3851b586af16bdf7346323bb046b0/cf2 is initiating minor compaction (all files) 2023-05-05 15:55:24,826 INFO [RS:0;jenkins-hbase4:43023-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 64a3851b586af16bdf7346323bb046b0/cf1 in testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0. 2023-05-05 15:55:24,826 INFO [RS:0;jenkins-hbase4:43023-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 64a3851b586af16bdf7346323bb046b0/cf2 in testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0. 2023-05-05 15:55:24,827 INFO [RS:0;jenkins-hbase4:43023-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/3f7a051f8fb3401a98e684e3f3c87774.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/3f7a051f8fb3401a98e684e3f3c87774-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/e7839c6630034b459434821a979cdba4.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/e7839c6630034b459434821a979cdba4-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/61ac5fa9999147bcbf73a6d18b5649cc.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/61ac5fa9999147bcbf73a6d18b5649cc-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/08fadfa3d55d45f584fc1bfc10edd030.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/08fadfa3d55d45f584fc1bfc10edd030-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/9df8d31606d24454a3f7dbc18cab3b82.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9df8d31606d24454a3f7dbc18cab3b82-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/6ea6afe5d7a04c7ebabc24c86506484b.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/6ea6afe5d7a04c7ebabc24c86506484b-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/9bdb81d2f49147a9a352edb000b4f39a.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9bdb81d2f49147a9a352edb000b4f39a-top] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/.tmp, totalSize=34.9 K 2023-05-05 15:55:24,827 INFO [RS:0;jenkins-hbase4:43023-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/4253289dd83c47bc97fc4d2a3d2ecf82.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/4253289dd83c47bc97fc4d2a3d2ecf82-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/0660dfdc5d6344eaadc077311ca6a079.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/0660dfdc5d6344eaadc077311ca6a079-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/7eade08af5844b98b3682905af5c9544.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7eade08af5844b98b3682905af5c9544-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/b13ef11746c94c8b87194ebbc346f8be.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/b13ef11746c94c8b87194ebbc346f8be-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/d5cc6bdc70b043298d32ca269e42816a.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/d5cc6bdc70b043298d32ca269e42816a-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/446e009084fb4a7e91db420c46a5a100.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/446e009084fb4a7e91db420c46a5a100-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/7569bf47fa6145f38deffbf61231360f.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7569bf47fa6145f38deffbf61231360f-top] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/.tmp, totalSize=34.9 K 2023-05-05 15:55:24,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0. 2023-05-05 15:55:24,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0. 2023-05-05 15:55:24,828 DEBUG [RS:0;jenkins-hbase4:43023-shortCompactions-0] compactions.Compactor(207): Compacting 3f7a051f8fb3401a98e684e3f3c87774.3faa7396b5050e4a3b1d1489c3bb7907, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683302120203 2023-05-05 15:55:24,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a. 2023-05-05 15:55:24,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c2b9dc0d93174d9e20cc3a4a80b2d71a, NAME => 'testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.', STARTKEY => '', ENDKEY => '41'} 2023-05-05 15:55:24,828 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=64a3851b586af16bdf7346323bb046b0, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:24,829 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683302124828"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302124828"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302124828"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302124828"}]},"ts":"1683302124828"} 2023-05-05 15:55:24,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:55:24,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution c2b9dc0d93174d9e20cc3a4a80b2d71a 2023-05-05 15:55:24,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:55:24,829 DEBUG [RS:0;jenkins-hbase4:43023-shortCompactions-0] compactions.Compactor(207): Compacting e7839c6630034b459434821a979cdba4.3faa7396b5050e4a3b1d1489c3bb7907, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683302120374 2023-05-05 15:55:24,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c2b9dc0d93174d9e20cc3a4a80b2d71a 2023-05-05 15:55:24,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c2b9dc0d93174d9e20cc3a4a80b2d71a 2023-05-05 15:55:24,830 DEBUG [RS:0;jenkins-hbase4:43023-longCompactions-0] compactions.Compactor(207): Compacting 4253289dd83c47bc97fc4d2a3d2ecf82.3faa7396b5050e4a3b1d1489c3bb7907, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683302120203 2023-05-05 15:55:24,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-05-05 15:55:24,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 64a3851b586af16bdf7346323bb046b0, server=jenkins-hbase4.apache.org,43023,1683302022922 in 234 msec 2023-05-05 15:55:24,832 DEBUG [RS:0;jenkins-hbase4:43023-shortCompactions-0] compactions.Compactor(207): Compacting 61ac5fa9999147bcbf73a6d18b5649cc.3faa7396b5050e4a3b1d1489c3bb7907, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683302120551 2023-05-05 15:55:24,832 DEBUG [RS:0;jenkins-hbase4:43023-longCompactions-0] compactions.Compactor(207): Compacting 0660dfdc5d6344eaadc077311ca6a079.3faa7396b5050e4a3b1d1489c3bb7907, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683302120374 2023-05-05 15:55:24,833 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=64a3851b586af16bdf7346323bb046b0, ASSIGN in 395 msec 2023-05-05 15:55:24,833 DEBUG [RS:0;jenkins-hbase4:43023-longCompactions-0] compactions.Compactor(207): Compacting 7eade08af5844b98b3682905af5c9544.3faa7396b5050e4a3b1d1489c3bb7907, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683302120551 2023-05-05 15:55:24,833 DEBUG [RS:0;jenkins-hbase4:43023-shortCompactions-0] compactions.Compactor(207): Compacting 08fadfa3d55d45f584fc1bfc10edd030.3faa7396b5050e4a3b1d1489c3bb7907, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1683302120727 2023-05-05 15:55:24,833 DEBUG [RS:0;jenkins-hbase4:43023-longCompactions-0] compactions.Compactor(207): Compacting b13ef11746c94c8b87194ebbc346f8be.3faa7396b5050e4a3b1d1489c3bb7907, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1683302120727 2023-05-05 15:55:24,834 INFO [StoreOpener-c2b9dc0d93174d9e20cc3a4a80b2d71a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c2b9dc0d93174d9e20cc3a4a80b2d71a 2023-05-05 15:55:24,834 DEBUG [RS:0;jenkins-hbase4:43023-shortCompactions-0] compactions.Compactor(207): Compacting 9df8d31606d24454a3f7dbc18cab3b82.3faa7396b5050e4a3b1d1489c3bb7907, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1683302120901 2023-05-05 15:55:24,834 DEBUG [RS:0;jenkins-hbase4:43023-longCompactions-0] compactions.Compactor(207): Compacting d5cc6bdc70b043298d32ca269e42816a.3faa7396b5050e4a3b1d1489c3bb7907, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1683302120901 2023-05-05 15:55:24,835 DEBUG [StoreOpener-c2b9dc0d93174d9e20cc3a4a80b2d71a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1 2023-05-05 15:55:24,835 DEBUG [StoreOpener-c2b9dc0d93174d9e20cc3a4a80b2d71a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1 2023-05-05 15:55:24,835 DEBUG [RS:0;jenkins-hbase4:43023-shortCompactions-0] compactions.Compactor(207): Compacting 6ea6afe5d7a04c7ebabc24c86506484b.3faa7396b5050e4a3b1d1489c3bb7907, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1683302121083 2023-05-05 15:55:24,835 INFO [StoreOpener-c2b9dc0d93174d9e20cc3a4a80b2d71a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c2b9dc0d93174d9e20cc3a4a80b2d71a columnFamilyName cf1 2023-05-05 15:55:24,835 DEBUG [RS:0;jenkins-hbase4:43023-longCompactions-0] compactions.Compactor(207): Compacting 446e009084fb4a7e91db420c46a5a100.3faa7396b5050e4a3b1d1489c3bb7907, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1683302121083 2023-05-05 15:55:24,836 DEBUG [RS:0;jenkins-hbase4:43023-shortCompactions-0] compactions.Compactor(207): Compacting 9bdb81d2f49147a9a352edb000b4f39a.3faa7396b5050e4a3b1d1489c3bb7907, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683302121256 2023-05-05 15:55:24,836 DEBUG [RS:0;jenkins-hbase4:43023-longCompactions-0] compactions.Compactor(207): Compacting 7569bf47fa6145f38deffbf61231360f.3faa7396b5050e4a3b1d1489c3bb7907, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683302121256 2023-05-05 15:55:24,851 INFO [RS:0;jenkins-hbase4:43023-longCompactions-0] throttle.PressureAwareThroughputController(145): 64a3851b586af16bdf7346323bb046b0#cf2#compaction#108 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:55:24,852 DEBUG [StoreOpener-c2b9dc0d93174d9e20cc3a4a80b2d71a-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/08fadfa3d55d45f584fc1bfc10edd030.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/08fadfa3d55d45f584fc1bfc10edd030-bottom 2023-05-05 15:55:24,861 INFO [RS:0;jenkins-hbase4:43023-shortCompactions-0] throttle.PressureAwareThroughputController(145): 64a3851b586af16bdf7346323bb046b0#cf1#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:55:24,863 DEBUG [StoreOpener-c2b9dc0d93174d9e20cc3a4a80b2d71a-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/3f7a051f8fb3401a98e684e3f3c87774.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/3f7a051f8fb3401a98e684e3f3c87774-bottom 2023-05-05 15:55:24,872 DEBUG [StoreOpener-c2b9dc0d93174d9e20cc3a4a80b2d71a-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/61ac5fa9999147bcbf73a6d18b5649cc.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/61ac5fa9999147bcbf73a6d18b5649cc-bottom 2023-05-05 15:55:24,877 DEBUG [StoreOpener-c2b9dc0d93174d9e20cc3a4a80b2d71a-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/6ea6afe5d7a04c7ebabc24c86506484b.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/6ea6afe5d7a04c7ebabc24c86506484b-bottom 2023-05-05 15:55:24,887 DEBUG [StoreOpener-c2b9dc0d93174d9e20cc3a4a80b2d71a-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/9bdb81d2f49147a9a352edb000b4f39a.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9bdb81d2f49147a9a352edb000b4f39a-bottom 2023-05-05 15:55:24,890 DEBUG [RS:0;jenkins-hbase4:43023-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/.tmp/cf2/6364511ea5d94d19b30e13989dbeb349 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/6364511ea5d94d19b30e13989dbeb349 2023-05-05 15:55:24,891 DEBUG [StoreOpener-c2b9dc0d93174d9e20cc3a4a80b2d71a-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/9df8d31606d24454a3f7dbc18cab3b82.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9df8d31606d24454a3f7dbc18cab3b82-bottom 2023-05-05 15:55:24,897 DEBUG [RS:0;jenkins-hbase4:43023-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/.tmp/cf1/0c294fb326824c4aa4d435f557f4ed7d as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/0c294fb326824c4aa4d435f557f4ed7d 2023-05-05 15:55:24,898 DEBUG [StoreOpener-c2b9dc0d93174d9e20cc3a4a80b2d71a-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/e7839c6630034b459434821a979cdba4.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/e7839c6630034b459434821a979cdba4-bottom 2023-05-05 15:55:24,900 INFO [StoreOpener-c2b9dc0d93174d9e20cc3a4a80b2d71a-1] regionserver.HStore(310): Store=c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:55:24,900 INFO [StoreOpener-c2b9dc0d93174d9e20cc3a4a80b2d71a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c2b9dc0d93174d9e20cc3a4a80b2d71a 2023-05-05 15:55:24,901 DEBUG [StoreOpener-c2b9dc0d93174d9e20cc3a4a80b2d71a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2 2023-05-05 15:55:24,901 DEBUG [StoreOpener-c2b9dc0d93174d9e20cc3a4a80b2d71a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2 2023-05-05 15:55:24,902 INFO [RS:0;jenkins-hbase4:43023-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 64a3851b586af16bdf7346323bb046b0/cf2 of 64a3851b586af16bdf7346323bb046b0 into 6364511ea5d94d19b30e13989dbeb349(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:55:24,902 INFO [StoreOpener-c2b9dc0d93174d9e20cc3a4a80b2d71a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c2b9dc0d93174d9e20cc3a4a80b2d71a columnFamilyName cf2 2023-05-05 15:55:24,902 DEBUG [RS:0;jenkins-hbase4:43023-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 64a3851b586af16bdf7346323bb046b0: 2023-05-05 15:55:24,902 INFO [RS:0;jenkins-hbase4:43023-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0., storeName=64a3851b586af16bdf7346323bb046b0/cf2, priority=9, startTime=1683302124823; duration=0sec 2023-05-05 15:55:24,902 DEBUG [RS:0;jenkins-hbase4:43023-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:55:24,903 INFO [RS:0;jenkins-hbase4:43023-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 64a3851b586af16bdf7346323bb046b0/cf1 of 64a3851b586af16bdf7346323bb046b0 into 0c294fb326824c4aa4d435f557f4ed7d(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:55:24,903 DEBUG [RS:0;jenkins-hbase4:43023-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 64a3851b586af16bdf7346323bb046b0: 2023-05-05 15:55:24,903 INFO [RS:0;jenkins-hbase4:43023-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0., storeName=64a3851b586af16bdf7346323bb046b0/cf1, priority=9, startTime=1683302124823; duration=0sec 2023-05-05 15:55:24,903 DEBUG [RS:0;jenkins-hbase4:43023-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:55:24,914 DEBUG [StoreOpener-c2b9dc0d93174d9e20cc3a4a80b2d71a-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/0660dfdc5d6344eaadc077311ca6a079.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/0660dfdc5d6344eaadc077311ca6a079-bottom 2023-05-05 15:55:24,917 DEBUG [StoreOpener-c2b9dc0d93174d9e20cc3a4a80b2d71a-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/4253289dd83c47bc97fc4d2a3d2ecf82.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/4253289dd83c47bc97fc4d2a3d2ecf82-bottom 2023-05-05 15:55:24,921 DEBUG [StoreOpener-c2b9dc0d93174d9e20cc3a4a80b2d71a-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/446e009084fb4a7e91db420c46a5a100.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/446e009084fb4a7e91db420c46a5a100-bottom 2023-05-05 15:55:24,925 DEBUG [StoreOpener-c2b9dc0d93174d9e20cc3a4a80b2d71a-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/7569bf47fa6145f38deffbf61231360f.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7569bf47fa6145f38deffbf61231360f-bottom 2023-05-05 15:55:24,930 DEBUG [StoreOpener-c2b9dc0d93174d9e20cc3a4a80b2d71a-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/7eade08af5844b98b3682905af5c9544.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7eade08af5844b98b3682905af5c9544-bottom 2023-05-05 15:55:24,933 DEBUG [StoreOpener-c2b9dc0d93174d9e20cc3a4a80b2d71a-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/b13ef11746c94c8b87194ebbc346f8be.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/b13ef11746c94c8b87194ebbc346f8be-bottom 2023-05-05 15:55:24,936 DEBUG [StoreOpener-c2b9dc0d93174d9e20cc3a4a80b2d71a-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/d5cc6bdc70b043298d32ca269e42816a.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/d5cc6bdc70b043298d32ca269e42816a-bottom 2023-05-05 15:55:24,936 INFO [StoreOpener-c2b9dc0d93174d9e20cc3a4a80b2d71a-1] regionserver.HStore(310): Store=c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:55:24,937 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a 2023-05-05 15:55:24,938 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a 2023-05-05 15:55:24,939 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-05 15:55:24,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c2b9dc0d93174d9e20cc3a4a80b2d71a 2023-05-05 15:55:24,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c2b9dc0d93174d9e20cc3a4a80b2d71a; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74313727, jitterRate=0.1073608249425888}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-05 15:55:24,941 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c2b9dc0d93174d9e20cc3a4a80b2d71a: 2023-05-05 15:55:24,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a., pid=117, masterSystemTime=1683302124744 2023-05-05 15:55:24,942 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:55:24,943 DEBUG [RS:0;jenkins-hbase4:43023-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-05 15:55:24,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:55:24,943 INFO [RS:0;jenkins-hbase4:43023-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a. 2023-05-05 15:55:24,943 DEBUG [RS:0;jenkins-hbase4:43023-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-05 15:55:24,943 DEBUG [RS:0;jenkins-hbase4:43023-shortCompactions-0] regionserver.HStore(1912): c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1 is initiating minor compaction (all files) 2023-05-05 15:55:24,943 INFO [RS:0;jenkins-hbase4:43023-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a. 2023-05-05 15:55:24,943 DEBUG [RS:0;jenkins-hbase4:43023-longCompactions-0] regionserver.HStore(1912): c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2 is initiating minor compaction (all files) 2023-05-05 15:55:24,943 INFO [RS:0;jenkins-hbase4:43023-longCompactions-0] regionserver.HRegion(2259): Starting compaction of c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2 in testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a. 2023-05-05 15:55:24,943 INFO [RS:0;jenkins-hbase4:43023-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1 in testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a. 2023-05-05 15:55:24,944 INFO [RS:0;jenkins-hbase4:43023-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/4253289dd83c47bc97fc4d2a3d2ecf82.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/4253289dd83c47bc97fc4d2a3d2ecf82-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/0660dfdc5d6344eaadc077311ca6a079.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/0660dfdc5d6344eaadc077311ca6a079-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/7eade08af5844b98b3682905af5c9544.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7eade08af5844b98b3682905af5c9544-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/b13ef11746c94c8b87194ebbc346f8be.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/b13ef11746c94c8b87194ebbc346f8be-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/d5cc6bdc70b043298d32ca269e42816a.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/d5cc6bdc70b043298d32ca269e42816a-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/446e009084fb4a7e91db420c46a5a100.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/446e009084fb4a7e91db420c46a5a100-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/7569bf47fa6145f38deffbf61231360f.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7569bf47fa6145f38deffbf61231360f-bottom] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/.tmp, totalSize=34.9 K 2023-05-05 15:55:24,944 INFO [RS:0;jenkins-hbase4:43023-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/3f7a051f8fb3401a98e684e3f3c87774.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/3f7a051f8fb3401a98e684e3f3c87774-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/e7839c6630034b459434821a979cdba4.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/e7839c6630034b459434821a979cdba4-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/61ac5fa9999147bcbf73a6d18b5649cc.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/61ac5fa9999147bcbf73a6d18b5649cc-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/08fadfa3d55d45f584fc1bfc10edd030.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/08fadfa3d55d45f584fc1bfc10edd030-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/9df8d31606d24454a3f7dbc18cab3b82.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9df8d31606d24454a3f7dbc18cab3b82-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/6ea6afe5d7a04c7ebabc24c86506484b.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/6ea6afe5d7a04c7ebabc24c86506484b-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/9bdb81d2f49147a9a352edb000b4f39a.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9bdb81d2f49147a9a352edb000b4f39a-bottom] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/.tmp, totalSize=34.9 K 2023-05-05 15:55:24,945 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a. 2023-05-05 15:55:24,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a. 2023-05-05 15:55:24,946 DEBUG [RS:0;jenkins-hbase4:43023-longCompactions-0] compactions.Compactor(207): Compacting 4253289dd83c47bc97fc4d2a3d2ecf82.3faa7396b5050e4a3b1d1489c3bb7907, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683302120203 2023-05-05 15:55:24,946 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=c2b9dc0d93174d9e20cc3a4a80b2d71a, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:24,946 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683302124946"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302124946"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302124946"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302124946"}]},"ts":"1683302124946"} 2023-05-05 15:55:24,947 DEBUG [RS:0;jenkins-hbase4:43023-longCompactions-0] compactions.Compactor(207): Compacting 0660dfdc5d6344eaadc077311ca6a079.3faa7396b5050e4a3b1d1489c3bb7907, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683302120374 2023-05-05 15:55:24,947 DEBUG [RS:0;jenkins-hbase4:43023-shortCompactions-0] compactions.Compactor(207): Compacting 3f7a051f8fb3401a98e684e3f3c87774.3faa7396b5050e4a3b1d1489c3bb7907, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683302120203 2023-05-05 15:55:24,948 DEBUG [RS:0;jenkins-hbase4:43023-longCompactions-0] compactions.Compactor(207): Compacting 7eade08af5844b98b3682905af5c9544.3faa7396b5050e4a3b1d1489c3bb7907, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683302120551 2023-05-05 15:55:24,948 DEBUG [RS:0;jenkins-hbase4:43023-shortCompactions-0] compactions.Compactor(207): Compacting e7839c6630034b459434821a979cdba4.3faa7396b5050e4a3b1d1489c3bb7907, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683302120374 2023-05-05 15:55:24,948 ERROR [BP-1744589926-172.31.14.131-1683302017433 heartbeating to localhost/127.0.0.1:35855] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1744589926-172.31.14.131-1683302017433 (Datanode Uuid 7884a1c8-f94e-4e8e-bd24-af43573a93c6) service to localhost/127.0.0.1:35855 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-05-05 15:55:24,949 DEBUG [RS:0;jenkins-hbase4:43023-shortCompactions-0] compactions.Compactor(207): Compacting 61ac5fa9999147bcbf73a6d18b5649cc.3faa7396b5050e4a3b1d1489c3bb7907, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683302120551 2023-05-05 15:55:24,949 DEBUG [RS:0;jenkins-hbase4:43023-longCompactions-0] compactions.Compactor(207): Compacting b13ef11746c94c8b87194ebbc346f8be.3faa7396b5050e4a3b1d1489c3bb7907, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683302120727 2023-05-05 15:55:24,949 DEBUG [RS:0;jenkins-hbase4:43023-shortCompactions-0] compactions.Compactor(207): Compacting 08fadfa3d55d45f584fc1bfc10edd030.3faa7396b5050e4a3b1d1489c3bb7907, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683302120727 2023-05-05 15:55:24,949 DEBUG [RS:0;jenkins-hbase4:43023-longCompactions-0] compactions.Compactor(207): Compacting d5cc6bdc70b043298d32ca269e42816a.3faa7396b5050e4a3b1d1489c3bb7907, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683302120901 2023-05-05 15:55:24,949 DEBUG [RS:0;jenkins-hbase4:43023-shortCompactions-0] compactions.Compactor(207): Compacting 9df8d31606d24454a3f7dbc18cab3b82.3faa7396b5050e4a3b1d1489c3bb7907, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683302120901 2023-05-05 15:55:24,950 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=115 2023-05-05 15:55:24,950 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure c2b9dc0d93174d9e20cc3a4a80b2d71a, server=jenkins-hbase4.apache.org,43023,1683302022922 in 355 msec 2023-05-05 15:55:24,950 DEBUG [RS:0;jenkins-hbase4:43023-shortCompactions-0] compactions.Compactor(207): Compacting 6ea6afe5d7a04c7ebabc24c86506484b.3faa7396b5050e4a3b1d1489c3bb7907, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683302121083 2023-05-05 15:55:24,951 DEBUG [RS:0;jenkins-hbase4:43023-longCompactions-0] compactions.Compactor(207): Compacting 446e009084fb4a7e91db420c46a5a100.3faa7396b5050e4a3b1d1489c3bb7907, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683302121083 2023-05-05 15:55:24,951 DEBUG [RS:0;jenkins-hbase4:43023-shortCompactions-0] compactions.Compactor(207): Compacting 9bdb81d2f49147a9a352edb000b4f39a.3faa7396b5050e4a3b1d1489c3bb7907, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683302121256 2023-05-05 15:55:24,951 DEBUG [RS:0;jenkins-hbase4:43023-longCompactions-0] compactions.Compactor(207): Compacting 7569bf47fa6145f38deffbf61231360f.3faa7396b5050e4a3b1d1489c3bb7907, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683302121256 2023-05-05 15:55:24,952 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=115, resume processing ppid=112 2023-05-05 15:55:24,952 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c2b9dc0d93174d9e20cc3a4a80b2d71a, ASSIGN in 513 msec 2023-05-05 15:55:24,952 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-05 15:55:24,952 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:24,952 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:55:24,952 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-05 15:55:24,963 INFO [RS:0;jenkins-hbase4:43023-longCompactions-0] throttle.PressureAwareThroughputController(145): c2b9dc0d93174d9e20cc3a4a80b2d71a#cf2#compaction#110 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:55:24,964 INFO [RS:0;jenkins-hbase4:43023-shortCompactions-0] throttle.PressureAwareThroughputController(145): c2b9dc0d93174d9e20cc3a4a80b2d71a#cf1#compaction#111 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:55:24,986 DEBUG [RS:0;jenkins-hbase4:43023-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/.tmp/cf1/b14de681b7a44acd9b069a5addc3b94f as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/b14de681b7a44acd9b069a5addc3b94f 2023-05-05 15:55:24,989 DEBUG [RS:0;jenkins-hbase4:43023-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/.tmp/cf2/6e06e96846b2402e8a544597e3d1bd66 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/6e06e96846b2402e8a544597e3d1bd66 2023-05-05 15:55:24,992 INFO [RS:0;jenkins-hbase4:43023-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1 of c2b9dc0d93174d9e20cc3a4a80b2d71a into b14de681b7a44acd9b069a5addc3b94f(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:55:24,992 DEBUG [RS:0;jenkins-hbase4:43023-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c2b9dc0d93174d9e20cc3a4a80b2d71a: 2023-05-05 15:55:24,992 INFO [RS:0;jenkins-hbase4:43023-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a., storeName=c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1, priority=9, startTime=1683302124942; duration=0sec 2023-05-05 15:55:24,992 DEBUG [RS:0;jenkins-hbase4:43023-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:55:24,995 INFO [RS:0;jenkins-hbase4:43023-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2 of c2b9dc0d93174d9e20cc3a4a80b2d71a into 6e06e96846b2402e8a544597e3d1bd66(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:55:24,995 DEBUG [RS:0;jenkins-hbase4:43023-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for c2b9dc0d93174d9e20cc3a4a80b2d71a: 2023-05-05 15:55:24,995 INFO [RS:0;jenkins-hbase4:43023-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a., storeName=c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2, priority=9, startTime=1683302124943; duration=0sec 2023-05-05 15:55:24,995 DEBUG [RS:0;jenkins-hbase4:43023-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:55:25,187 INFO [Listener at localhost/35117] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:25,187 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-05 15:55:25,187 INFO [Listener at localhost/35117] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:55:25,188 INFO [Listener at localhost/35117] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:55:25,188 ERROR [Listener at localhost/35117] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-05 15:55:25,188 INFO [Listener at localhost/35117] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-05 15:55:25,188 INFO [Listener at localhost/35117] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-05 15:55:25,188 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-05 15:55:25,188 INFO [Listener at localhost/35117] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-05 15:55:25,189 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-05 15:55:25,190 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-05-05 15:55:25,190 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-05 15:55:25,190 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-05 15:55:25,190 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 16, 23] 2023-05-05 15:55:25,190 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 22] 2023-05-05 15:55:25,190 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 18, 21] 2023-05-05 15:55:25,190 DEBUG [Listener at localhost/35117] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 20] 2023-05-05 15:55:25,190 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-05 15:55:25,191 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-05 15:55:25,191 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-05 15:55:25,191 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-05 15:55:25,191 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:25,191 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=2faa234e4e385b82c11337930631d2a1, daughterA=e24ba92d7211d01fc6da0d90d6b8b4e1, daughterB=7ba5bebcc9ff92eb8e9c5a4ad2ecf2f9 2023-05-05 15:55:25,192 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:25,192 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-05 15:55:25,192 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:25,192 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=82d51cc6743c70579253652610cbc613, daughterA=6f0ed874c66cda1037a613cc845c4e7e, daughterB=e800c4983abdf0cef46c4396eec67564 2023-05-05 15:55:25,192 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:25,192 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-05 15:55:25,192 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:25,193 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=b299eb879fd7ed0f63e4b91ba5268da3, daughterA=b13dfdeca1ae2cc48d85f970e4c58cc2, daughterB=e73cfd57ec9ec904b48f85318b83d6ac 2023-05-05 15:55:25,193 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=b299eb879fd7ed0f63e4b91ba5268da3, ASSIGN 2023-05-05 15:55:25,193 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:25,193 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-05 15:55:25,193 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:25,193 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=cfa9eb043f55be3f9ab828cd40416e7f, daughterA=3296930d4aa28614ec2511a0a3f26d3c, daughterB=c4477a15aabbb6987e68dcac21577a7f 2023-05-05 15:55:25,193 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:25,194 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-05 15:55:25,194 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:25,194 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=df08d89907b6ac6a28b72ec117993ac2, daughterA=6cb2bd681dce908bd0c281b2c71d45e8, daughterB=2d9826f21eb94a559f0d5b68d98cb98d 2023-05-05 15:55:25,194 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:25,194 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-05 15:55:25,194 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:25,195 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:25,195 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42799,1683302022977, table=testRollbackForSplitTableRegionWithReplica, region=47d3ff38207fecb2cbad8f4f5aab825b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef3937c5a4e99c77bfe5f10af1ca890, daughterA=b385b26868fcaaccb79e222957786618, daughterB=a13170d1b393d49a10002f92ce6dcd48 2023-05-05 15:55:25,195 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:25,195 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-05 15:55:25,195 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:25,196 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:25,196 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-05 15:55:25,196 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-05 15:55:25,197 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:25,198 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907, UNASSIGN 2023-05-05 15:55:25,198 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3faa7396b5050e4a3b1d1489c3bb7907, server=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:25,198 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c2b9dc0d93174d9e20cc3a4a80b2d71a, ASSIGN 2023-05-05 15:55:25,198 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=64a3851b586af16bdf7346323bb046b0, ASSIGN 2023-05-05 15:55:25,198 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure c2b9dc0d93174d9e20cc3a4a80b2d71a, server=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:25,198 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 64a3851b586af16bdf7346323bb046b0, server=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:25,198 DEBUG [Listener at localhost/35117] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 held the lock before restarting, call acquireLock to restore it. 2023-05-05 15:55:25,198 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:25,198 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:25,199 INFO [Listener at localhost/35117] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:25,199 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:55:25,199 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3faa7396b5050e4a3b1d1489c3bb7907, server=jenkins-hbase4.apache.org,43023,1683302022922 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:55:25,199 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c2b9dc0d93174d9e20cc3a4a80b2d71a, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:55:25,199 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=64a3851b586af16bdf7346323bb046b0, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:55:25,199 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure c2b9dc0d93174d9e20cc3a4a80b2d71a, server=jenkins-hbase4.apache.org,43023,1683302022922 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:55:25,199 DEBUG [Listener at localhost/35117] procedure2.Procedure(946): pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 64a3851b586af16bdf7346323bb046b0, server=jenkins-hbase4.apache.org,43023,1683302022922 didn't hold the lock before restarting, skip acquiring lock. 2023-05-05 15:55:25,199 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-05-05 15:55:25,200 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-05 15:55:25,201 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016, table=hbase:meta, region=1588230740 2023-05-05 15:55:25,201 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683302155201 2023-05-05 15:55:25,203 DEBUG [Listener at localhost/35117] assignment.AssignmentManager(1517): Joining cluster... 2023-05-05 15:55:25,203 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:25,206 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=d05f6ba26fbb949a8331f3129bf8461a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:55:25,206 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=3faa7396b5050e4a3b1d1489c3bb7907, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=2 2023-05-05 15:55:25,206 INFO [Listener at localhost/35117] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,43023,1683302022922, table=testRecoveryAndDoubleExecution, region=3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,207 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=c2b9dc0d93174d9e20cc3a4a80b2d71a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=86 2023-05-05 15:55:25,207 DEBUG [Listener at localhost/35117] assignment.RegionStateStore(147): Load hbase:meta entry region=64a3851b586af16bdf7346323bb046b0, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43023,1683302022922, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922, openSeqNum=86 2023-05-05 15:55:25,207 INFO [Listener at localhost/35117] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-05 15:55:25,207 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683302185207 2023-05-05 15:55:25,207 INFO [Listener at localhost/35117] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683302245207 2023-05-05 15:55:25,207 INFO [Listener at localhost/35117] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-05 15:55:25,207 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-05 15:55:25,210 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3faa7396b5050e4a3b1d1489c3bb7907, daughterA=c2b9dc0d93174d9e20cc3a4a80b2d71a, daughterB=64a3851b586af16bdf7346323bb046b0 in 3.6930 sec 2023-05-05 15:55:25,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43023] ipc.CallRunner(144): callId: 860 service: ClientService methodName: Scan size: 154 connection: 172.31.14.131:58214 deadline: 1683302185458, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907. is not online on jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:25,569 DEBUG [Listener at localhost/35117] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-05 15:55:25,574 INFO [Listener at localhost/35117] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-05-05 15:55:25,576 DEBUG [Listener at localhost/35117] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-05 15:55:25,583 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-05 15:55:25,585 INFO [Listener at localhost/35117] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-05-05 15:55:25,585 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRecoveryAndDoubleExecution 2023-05-05 15:55:25,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-05-05 15:55:25,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-05-05 15:55:25,588 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302125588"}]},"ts":"1683302125588"} 2023-05-05 15:55:25,589 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-05-05 15:55:25,591 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-05-05 15:55:25,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c2b9dc0d93174d9e20cc3a4a80b2d71a, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=64a3851b586af16bdf7346323bb046b0, UNASSIGN}] 2023-05-05 15:55:25,592 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=64a3851b586af16bdf7346323bb046b0, UNASSIGN 2023-05-05 15:55:25,592 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=64a3851b586af16bdf7346323bb046b0, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:25,592 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683302125592"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302125592"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302125592"}]},"ts":"1683302125592"} 2023-05-05 15:55:25,593 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 64a3851b586af16bdf7346323bb046b0, server=jenkins-hbase4.apache.org,43023,1683302022922}] 2023-05-05 15:55:25,595 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c2b9dc0d93174d9e20cc3a4a80b2d71a, UNASSIGN 2023-05-05 15:55:25,595 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=c2b9dc0d93174d9e20cc3a4a80b2d71a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:25,595 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683302125595"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302125595"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302125595"}]},"ts":"1683302125595"} 2023-05-05 15:55:25,596 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure c2b9dc0d93174d9e20cc3a4a80b2d71a, server=jenkins-hbase4.apache.org,43023,1683302022922}] 2023-05-05 15:55:25,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-05-05 15:55:25,745 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close c2b9dc0d93174d9e20cc3a4a80b2d71a 2023-05-05 15:55:25,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c2b9dc0d93174d9e20cc3a4a80b2d71a, disabling compactions & flushes 2023-05-05 15:55:25,745 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a. 2023-05-05 15:55:25,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a. 2023-05-05 15:55:25,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a. after waiting 0 ms 2023-05-05 15:55:25,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a. 2023-05-05 15:55:25,746 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/3f7a051f8fb3401a98e684e3f3c87774.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/3f7a051f8fb3401a98e684e3f3c87774-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/e7839c6630034b459434821a979cdba4.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/e7839c6630034b459434821a979cdba4-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/61ac5fa9999147bcbf73a6d18b5649cc.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/61ac5fa9999147bcbf73a6d18b5649cc-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/08fadfa3d55d45f584fc1bfc10edd030.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/08fadfa3d55d45f584fc1bfc10edd030-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/9df8d31606d24454a3f7dbc18cab3b82.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9df8d31606d24454a3f7dbc18cab3b82-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/6ea6afe5d7a04c7ebabc24c86506484b.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/6ea6afe5d7a04c7ebabc24c86506484b-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/9bdb81d2f49147a9a352edb000b4f39a.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9bdb81d2f49147a9a352edb000b4f39a-bottom] to archive 2023-05-05 15:55:25,747 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-05 15:55:25,748 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/3f7a051f8fb3401a98e684e3f3c87774.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/3f7a051f8fb3401a98e684e3f3c87774.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,750 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/e7839c6630034b459434821a979cdba4.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/e7839c6630034b459434821a979cdba4.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,751 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/61ac5fa9999147bcbf73a6d18b5649cc.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/61ac5fa9999147bcbf73a6d18b5649cc.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,752 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/08fadfa3d55d45f584fc1bfc10edd030.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/08fadfa3d55d45f584fc1bfc10edd030.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,753 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/9df8d31606d24454a3f7dbc18cab3b82.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/9df8d31606d24454a3f7dbc18cab3b82.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,754 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/6ea6afe5d7a04c7ebabc24c86506484b.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/6ea6afe5d7a04c7ebabc24c86506484b.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,756 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/9bdb81d2f49147a9a352edb000b4f39a.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/9bdb81d2f49147a9a352edb000b4f39a.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,757 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/4253289dd83c47bc97fc4d2a3d2ecf82.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/4253289dd83c47bc97fc4d2a3d2ecf82-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/0660dfdc5d6344eaadc077311ca6a079.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/0660dfdc5d6344eaadc077311ca6a079-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/7eade08af5844b98b3682905af5c9544.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7eade08af5844b98b3682905af5c9544-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/b13ef11746c94c8b87194ebbc346f8be.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/b13ef11746c94c8b87194ebbc346f8be-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/d5cc6bdc70b043298d32ca269e42816a.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/d5cc6bdc70b043298d32ca269e42816a-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/446e009084fb4a7e91db420c46a5a100.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/446e009084fb4a7e91db420c46a5a100-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/7569bf47fa6145f38deffbf61231360f.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7569bf47fa6145f38deffbf61231360f-bottom] to archive 2023-05-05 15:55:25,758 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-05 15:55:25,760 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/4253289dd83c47bc97fc4d2a3d2ecf82.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/4253289dd83c47bc97fc4d2a3d2ecf82.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,761 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/0660dfdc5d6344eaadc077311ca6a079.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/0660dfdc5d6344eaadc077311ca6a079.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,762 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/7eade08af5844b98b3682905af5c9544.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/7eade08af5844b98b3682905af5c9544.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,763 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/b13ef11746c94c8b87194ebbc346f8be.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/b13ef11746c94c8b87194ebbc346f8be.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,764 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/d5cc6bdc70b043298d32ca269e42816a.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/d5cc6bdc70b043298d32ca269e42816a.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,766 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/446e009084fb4a7e91db420c46a5a100.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/446e009084fb4a7e91db420c46a5a100.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,767 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/7569bf47fa6145f38deffbf61231360f.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/7569bf47fa6145f38deffbf61231360f.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,770 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-05 15:55:25,770 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:55:25,771 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a. 2023-05-05 15:55:25,771 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c2b9dc0d93174d9e20cc3a4a80b2d71a: 2023-05-05 15:55:25,773 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed c2b9dc0d93174d9e20cc3a4a80b2d71a 2023-05-05 15:55:25,773 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:25,773 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 64a3851b586af16bdf7346323bb046b0, disabling compactions & flushes 2023-05-05 15:55:25,773 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0. 2023-05-05 15:55:25,773 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0. 2023-05-05 15:55:25,773 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0. after waiting 0 ms 2023-05-05 15:55:25,773 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0. 2023-05-05 15:55:25,773 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=c2b9dc0d93174d9e20cc3a4a80b2d71a, regionState=CLOSED 2023-05-05 15:55:25,773 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683302125773"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302125773"}]},"ts":"1683302125773"} 2023-05-05 15:55:25,774 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/3f7a051f8fb3401a98e684e3f3c87774.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/3f7a051f8fb3401a98e684e3f3c87774-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/e7839c6630034b459434821a979cdba4.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/e7839c6630034b459434821a979cdba4-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/61ac5fa9999147bcbf73a6d18b5649cc.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/61ac5fa9999147bcbf73a6d18b5649cc-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/08fadfa3d55d45f584fc1bfc10edd030.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/08fadfa3d55d45f584fc1bfc10edd030-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/9df8d31606d24454a3f7dbc18cab3b82.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9df8d31606d24454a3f7dbc18cab3b82-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/6ea6afe5d7a04c7ebabc24c86506484b.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/6ea6afe5d7a04c7ebabc24c86506484b-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/9bdb81d2f49147a9a352edb000b4f39a.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9bdb81d2f49147a9a352edb000b4f39a-top] to archive 2023-05-05 15:55:25,775 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-05 15:55:25,776 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-05-05 15:55:25,776 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure c2b9dc0d93174d9e20cc3a4a80b2d71a, server=jenkins-hbase4.apache.org,43023,1683302022922 in 179 msec 2023-05-05 15:55:25,776 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/3f7a051f8fb3401a98e684e3f3c87774.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/3f7a051f8fb3401a98e684e3f3c87774.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,777 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c2b9dc0d93174d9e20cc3a4a80b2d71a, UNASSIGN in 185 msec 2023-05-05 15:55:25,778 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/e7839c6630034b459434821a979cdba4.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/e7839c6630034b459434821a979cdba4.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,778 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/61ac5fa9999147bcbf73a6d18b5649cc.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/61ac5fa9999147bcbf73a6d18b5649cc.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,780 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/08fadfa3d55d45f584fc1bfc10edd030.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/08fadfa3d55d45f584fc1bfc10edd030.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,781 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/9df8d31606d24454a3f7dbc18cab3b82.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/9df8d31606d24454a3f7dbc18cab3b82.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,781 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/6ea6afe5d7a04c7ebabc24c86506484b.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/6ea6afe5d7a04c7ebabc24c86506484b.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,782 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/9bdb81d2f49147a9a352edb000b4f39a.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/9bdb81d2f49147a9a352edb000b4f39a.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,784 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/4253289dd83c47bc97fc4d2a3d2ecf82.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/4253289dd83c47bc97fc4d2a3d2ecf82-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/0660dfdc5d6344eaadc077311ca6a079.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/0660dfdc5d6344eaadc077311ca6a079-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/7eade08af5844b98b3682905af5c9544.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7eade08af5844b98b3682905af5c9544-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/b13ef11746c94c8b87194ebbc346f8be.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/b13ef11746c94c8b87194ebbc346f8be-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/d5cc6bdc70b043298d32ca269e42816a.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/d5cc6bdc70b043298d32ca269e42816a-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/446e009084fb4a7e91db420c46a5a100.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/446e009084fb4a7e91db420c46a5a100-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/7569bf47fa6145f38deffbf61231360f.3faa7396b5050e4a3b1d1489c3bb7907->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7569bf47fa6145f38deffbf61231360f-top] to archive 2023-05-05 15:55:25,784 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-05 15:55:25,786 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/4253289dd83c47bc97fc4d2a3d2ecf82.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/4253289dd83c47bc97fc4d2a3d2ecf82.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,786 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/0660dfdc5d6344eaadc077311ca6a079.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/0660dfdc5d6344eaadc077311ca6a079.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,787 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/7eade08af5844b98b3682905af5c9544.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/7eade08af5844b98b3682905af5c9544.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,788 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/b13ef11746c94c8b87194ebbc346f8be.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/b13ef11746c94c8b87194ebbc346f8be.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,789 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/d5cc6bdc70b043298d32ca269e42816a.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/d5cc6bdc70b043298d32ca269e42816a.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,790 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/446e009084fb4a7e91db420c46a5a100.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/446e009084fb4a7e91db420c46a5a100.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,791 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/7569bf47fa6145f38deffbf61231360f.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/7569bf47fa6145f38deffbf61231360f.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,795 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-05 15:55:25,795 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:55:25,795 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0. 2023-05-05 15:55:25,795 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 64a3851b586af16bdf7346323bb046b0: 2023-05-05 15:55:25,797 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:25,798 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=64a3851b586af16bdf7346323bb046b0, regionState=CLOSED 2023-05-05 15:55:25,798 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683302125798"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302125798"}]},"ts":"1683302125798"} 2023-05-05 15:55:25,800 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-05-05 15:55:25,800 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 64a3851b586af16bdf7346323bb046b0, server=jenkins-hbase4.apache.org,43023,1683302022922 in 206 msec 2023-05-05 15:55:25,801 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=119 2023-05-05 15:55:25,801 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=64a3851b586af16bdf7346323bb046b0, UNASSIGN in 209 msec 2023-05-05 15:55:25,802 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302125802"}]},"ts":"1683302125802"} 2023-05-05 15:55:25,802 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-05-05 15:55:25,804 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-05-05 15:55:25,805 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 219 msec 2023-05-05 15:55:25,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-05-05 15:55:25,890 INFO [Listener at localhost/35117] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-05-05 15:55:25,891 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRecoveryAndDoubleExecution 2023-05-05 15:55:25,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-05-05 15:55:25,893 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-05-05 15:55:25,894 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-05-05 15:55:25,894 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-05-05 15:55:25,898 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,898 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a 2023-05-05 15:55:25,898 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:25,900 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/recovered.edits] 2023-05-05 15:55:25,901 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/recovered.edits] 2023-05-05 15:55:25,902 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/recovered.edits] 2023-05-05 15:55:25,904 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/.regioninfo to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/.regioninfo 2023-05-05 15:55:25,905 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/08fadfa3d55d45f584fc1bfc10edd030 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/08fadfa3d55d45f584fc1bfc10edd030 2023-05-05 15:55:25,905 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/.regioninfo to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/.regioninfo 2023-05-05 15:55:25,906 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/3f7a051f8fb3401a98e684e3f3c87774 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/3f7a051f8fb3401a98e684e3f3c87774 2023-05-05 15:55:25,907 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/08fadfa3d55d45f584fc1bfc10edd030.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/08fadfa3d55d45f584fc1bfc10edd030.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,907 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/61ac5fa9999147bcbf73a6d18b5649cc to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/61ac5fa9999147bcbf73a6d18b5649cc 2023-05-05 15:55:25,908 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf1/08fadfa3d55d45f584fc1bfc10edd030.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf1/08fadfa3d55d45f584fc1bfc10edd030.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,909 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/3f7a051f8fb3401a98e684e3f3c87774.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/3f7a051f8fb3401a98e684e3f3c87774.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,909 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/6ea6afe5d7a04c7ebabc24c86506484b to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/6ea6afe5d7a04c7ebabc24c86506484b 2023-05-05 15:55:25,910 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf1/3f7a051f8fb3401a98e684e3f3c87774.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf1/3f7a051f8fb3401a98e684e3f3c87774.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,910 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9bdb81d2f49147a9a352edb000b4f39a to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9bdb81d2f49147a9a352edb000b4f39a 2023-05-05 15:55:25,910 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/61ac5fa9999147bcbf73a6d18b5649cc.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/61ac5fa9999147bcbf73a6d18b5649cc.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,911 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf1/61ac5fa9999147bcbf73a6d18b5649cc.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf1/61ac5fa9999147bcbf73a6d18b5649cc.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,911 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9df8d31606d24454a3f7dbc18cab3b82 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/9df8d31606d24454a3f7dbc18cab3b82 2023-05-05 15:55:25,912 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/6ea6afe5d7a04c7ebabc24c86506484b.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/6ea6afe5d7a04c7ebabc24c86506484b.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,913 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf1/6ea6afe5d7a04c7ebabc24c86506484b.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf1/6ea6afe5d7a04c7ebabc24c86506484b.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,913 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/9bdb81d2f49147a9a352edb000b4f39a.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/9bdb81d2f49147a9a352edb000b4f39a.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,913 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/e7839c6630034b459434821a979cdba4 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf1/e7839c6630034b459434821a979cdba4 2023-05-05 15:55:25,914 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf1/9bdb81d2f49147a9a352edb000b4f39a.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf1/9bdb81d2f49147a9a352edb000b4f39a.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,914 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/9df8d31606d24454a3f7dbc18cab3b82.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/9df8d31606d24454a3f7dbc18cab3b82.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,917 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/0660dfdc5d6344eaadc077311ca6a079 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/0660dfdc5d6344eaadc077311ca6a079 2023-05-05 15:55:25,917 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/e7839c6630034b459434821a979cdba4.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/e7839c6630034b459434821a979cdba4.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,917 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf1/9df8d31606d24454a3f7dbc18cab3b82.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf1/9df8d31606d24454a3f7dbc18cab3b82.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,919 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/4253289dd83c47bc97fc4d2a3d2ecf82 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/4253289dd83c47bc97fc4d2a3d2ecf82 2023-05-05 15:55:25,919 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf1/e7839c6630034b459434821a979cdba4.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf1/e7839c6630034b459434821a979cdba4.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,920 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/446e009084fb4a7e91db420c46a5a100 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/446e009084fb4a7e91db420c46a5a100 2023-05-05 15:55:25,921 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/0660dfdc5d6344eaadc077311ca6a079.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/0660dfdc5d6344eaadc077311ca6a079.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,921 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7569bf47fa6145f38deffbf61231360f to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7569bf47fa6145f38deffbf61231360f 2023-05-05 15:55:25,922 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/4253289dd83c47bc97fc4d2a3d2ecf82.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/4253289dd83c47bc97fc4d2a3d2ecf82.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,922 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7eade08af5844b98b3682905af5c9544 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/7eade08af5844b98b3682905af5c9544 2023-05-05 15:55:25,923 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf2/0660dfdc5d6344eaadc077311ca6a079.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf2/0660dfdc5d6344eaadc077311ca6a079.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,930 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/446e009084fb4a7e91db420c46a5a100.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/446e009084fb4a7e91db420c46a5a100.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,930 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf2/4253289dd83c47bc97fc4d2a3d2ecf82.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf2/4253289dd83c47bc97fc4d2a3d2ecf82.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,930 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/b13ef11746c94c8b87194ebbc346f8be to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/b13ef11746c94c8b87194ebbc346f8be 2023-05-05 15:55:25,932 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/7569bf47fa6145f38deffbf61231360f.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/7569bf47fa6145f38deffbf61231360f.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,932 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/d5cc6bdc70b043298d32ca269e42816a to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/cf2/d5cc6bdc70b043298d32ca269e42816a 2023-05-05 15:55:25,932 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf2/446e009084fb4a7e91db420c46a5a100.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf2/446e009084fb4a7e91db420c46a5a100.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,934 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/7eade08af5844b98b3682905af5c9544.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/7eade08af5844b98b3682905af5c9544.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,934 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf2/7569bf47fa6145f38deffbf61231360f.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf2/7569bf47fa6145f38deffbf61231360f.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,936 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/recovered.edits/85.seqid to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907/recovered.edits/85.seqid 2023-05-05 15:55:25,936 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf2/7eade08af5844b98b3682905af5c9544.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf2/7eade08af5844b98b3682905af5c9544.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,936 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/b13ef11746c94c8b87194ebbc346f8be.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/b13ef11746c94c8b87194ebbc346f8be.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,936 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,937 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf2/b13ef11746c94c8b87194ebbc346f8be.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf2/b13ef11746c94c8b87194ebbc346f8be.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,937 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/d5cc6bdc70b043298d32ca269e42816a.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/d5cc6bdc70b043298d32ca269e42816a.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,938 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf2/d5cc6bdc70b043298d32ca269e42816a.3faa7396b5050e4a3b1d1489c3bb7907 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/64a3851b586af16bdf7346323bb046b0/cf2/d5cc6bdc70b043298d32ca269e42816a.3faa7396b5050e4a3b1d1489c3bb7907 2023-05-05 15:55:25,939 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/b14de681b7a44acd9b069a5addc3b94f to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf1/b14de681b7a44acd9b069a5addc3b94f 2023-05-05 15:55:25,940 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/0c294fb326824c4aa4d435f557f4ed7d to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf1/0c294fb326824c4aa4d435f557f4ed7d 2023-05-05 15:55:25,942 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/6e06e96846b2402e8a544597e3d1bd66 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/cf2/6e06e96846b2402e8a544597e3d1bd66 2023-05-05 15:55:25,942 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/6364511ea5d94d19b30e13989dbeb349 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/cf2/6364511ea5d94d19b30e13989dbeb349 2023-05-05 15:55:25,944 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/recovered.edits/90.seqid to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a/recovered.edits/90.seqid 2023-05-05 15:55:25,945 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/c2b9dc0d93174d9e20cc3a4a80b2d71a 2023-05-05 15:55:25,945 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/recovered.edits/90.seqid to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0/recovered.edits/90.seqid 2023-05-05 15:55:25,945 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testRecoveryAndDoubleExecution/64a3851b586af16bdf7346323bb046b0 2023-05-05 15:55:25,945 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-05-05 15:55:25,947 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-05-05 15:55:25,949 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-05-05 15:55:25,950 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-05-05 15:55:25,951 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-05-05 15:55:25,951 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-05-05 15:55:25,951 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302125951"}]},"ts":"9223372036854775807"} 2023-05-05 15:55:25,951 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302125951"}]},"ts":"9223372036854775807"} 2023-05-05 15:55:25,951 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302125951"}]},"ts":"9223372036854775807"} 2023-05-05 15:55:25,953 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-05 15:55:25,953 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 3faa7396b5050e4a3b1d1489c3bb7907, NAME => 'testRecoveryAndDoubleExecution,,1683302119700.3faa7396b5050e4a3b1d1489c3bb7907.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => c2b9dc0d93174d9e20cc3a4a80b2d71a, NAME => 'testRecoveryAndDoubleExecution,,1683302121514.c2b9dc0d93174d9e20cc3a4a80b2d71a.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 64a3851b586af16bdf7346323bb046b0, NAME => 'testRecoveryAndDoubleExecution,41,1683302121514.64a3851b586af16bdf7346323bb046b0.', STARTKEY => '41', ENDKEY => ''}] 2023-05-05 15:55:25,953 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-05-05 15:55:25,953 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683302125953"}]},"ts":"9223372036854775807"} 2023-05-05 15:55:25,954 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-05-05 15:55:25,962 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-05-05 15:55:25,963 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 71 msec 2023-05-05 15:55:25,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-05-05 15:55:25,995 INFO [Listener at localhost/35117] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-05-05 15:55:26,010 INFO [Listener at localhost/35117] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=377 (was 368) - Thread LEAK? -, OpenFileDescriptor=859 (was 742) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=275 (was 186) - SystemLoadAverage LEAK? -, ProcessCount=179 (was 179), AvailableMemoryMB=5111 (was 5122) 2023-05-05 15:55:26,024 INFO [Listener at localhost/35117] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=377, OpenFileDescriptor=859, MaxFileDescriptor=60000, SystemLoadAverage=275, ProcessCount=179, AvailableMemoryMB=5110 2023-05-05 15:55:26,024 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-05 15:55:26,026 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-05 15:55:26,026 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-05 15:55:26,027 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,029 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-05-05 15:55:26,029 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-05 15:55:26,031 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:26,031 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c empty. 2023-05-05 15:55:26,031 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:26,031 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-05-05 15:55:26,040 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-05-05 15:55:26,041 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => 94b959e0a954a0b5ed4ceaeaa080b44c, NAME => 'testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp 2023-05-05 15:55:26,048 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:55:26,048 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1604): Closing 94b959e0a954a0b5ed4ceaeaa080b44c, disabling compactions & flushes 2023-05-05 15:55:26,048 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. 2023-05-05 15:55:26,048 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. 2023-05-05 15:55:26,048 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. after waiting 0 ms 2023-05-05 15:55:26,048 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. 2023-05-05 15:55:26,048 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. 2023-05-05 15:55:26,048 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1558): Region close journal for 94b959e0a954a0b5ed4ceaeaa080b44c: 2023-05-05 15:55:26,050 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-05-05 15:55:26,051 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683302126050"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302126050"}]},"ts":"1683302126050"} 2023-05-05 15:55:26,052 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-05 15:55:26,052 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-05 15:55:26,052 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302126052"}]},"ts":"1683302126052"} 2023-05-05 15:55:26,053 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-05-05 15:55:26,056 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-05 15:55:26,057 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-05 15:55:26,057 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-05 15:55:26,057 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-05 15:55:26,057 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-05 15:55:26,057 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=94b959e0a954a0b5ed4ceaeaa080b44c, ASSIGN}] 2023-05-05 15:55:26,059 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=94b959e0a954a0b5ed4ceaeaa080b44c, ASSIGN 2023-05-05 15:55:26,059 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=94b959e0a954a0b5ed4ceaeaa080b44c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40991,1683302023016; forceNewPlan=false, retain=false 2023-05-05 15:55:26,209 INFO [jenkins-hbase4:44213] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-05 15:55:26,210 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=94b959e0a954a0b5ed4ceaeaa080b44c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,211 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683302126210"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302126210"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302126210"}]},"ts":"1683302126210"} 2023-05-05 15:55:26,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 94b959e0a954a0b5ed4ceaeaa080b44c, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:55:26,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. 2023-05-05 15:55:26,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 94b959e0a954a0b5ed4ceaeaa080b44c, NAME => 'testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c.', STARTKEY => '', ENDKEY => ''} 2023-05-05 15:55:26,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:55:26,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:26,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:55:26,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:26,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:26,368 INFO [StoreOpener-94b959e0a954a0b5ed4ceaeaa080b44c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:26,369 DEBUG [StoreOpener-94b959e0a954a0b5ed4ceaeaa080b44c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1 2023-05-05 15:55:26,369 DEBUG [StoreOpener-94b959e0a954a0b5ed4ceaeaa080b44c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1 2023-05-05 15:55:26,370 INFO [StoreOpener-94b959e0a954a0b5ed4ceaeaa080b44c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 94b959e0a954a0b5ed4ceaeaa080b44c columnFamilyName cf1 2023-05-05 15:55:26,370 INFO [StoreOpener-94b959e0a954a0b5ed4ceaeaa080b44c-1] regionserver.HStore(310): Store=94b959e0a954a0b5ed4ceaeaa080b44c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:55:26,370 INFO [StoreOpener-94b959e0a954a0b5ed4ceaeaa080b44c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:26,371 DEBUG [StoreOpener-94b959e0a954a0b5ed4ceaeaa080b44c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2 2023-05-05 15:55:26,371 DEBUG [StoreOpener-94b959e0a954a0b5ed4ceaeaa080b44c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2 2023-05-05 15:55:26,371 INFO [StoreOpener-94b959e0a954a0b5ed4ceaeaa080b44c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 94b959e0a954a0b5ed4ceaeaa080b44c columnFamilyName cf2 2023-05-05 15:55:26,372 INFO [StoreOpener-94b959e0a954a0b5ed4ceaeaa080b44c-1] regionserver.HStore(310): Store=94b959e0a954a0b5ed4ceaeaa080b44c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:55:26,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:26,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:26,374 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-05 15:55:26,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:26,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-05 15:55:26,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 94b959e0a954a0b5ed4ceaeaa080b44c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59586962, jitterRate=-0.11208507418632507}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-05 15:55:26,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 94b959e0a954a0b5ed4ceaeaa080b44c: 2023-05-05 15:55:26,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c., pid=127, masterSystemTime=1683302126364 2023-05-05 15:55:26,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. 2023-05-05 15:55:26,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. 2023-05-05 15:55:26,379 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=94b959e0a954a0b5ed4ceaeaa080b44c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,380 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683302126379"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302126379"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302126379"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302126379"}]},"ts":"1683302126379"} 2023-05-05 15:55:26,383 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-05-05 15:55:26,383 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 94b959e0a954a0b5ed4ceaeaa080b44c, server=jenkins-hbase4.apache.org,40991,1683302023016 in 169 msec 2023-05-05 15:55:26,384 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-05-05 15:55:26,384 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=94b959e0a954a0b5ed4ceaeaa080b44c, ASSIGN in 326 msec 2023-05-05 15:55:26,385 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-05 15:55:26,385 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302126385"}]},"ts":"1683302126385"} 2023-05-05 15:55:26,386 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-05-05 15:55:26,389 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-05-05 15:55:26,390 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 362 msec 2023-05-05 15:55:26,534 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:55:26,536 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,536 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-05-05 15:55:26,536 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:26,536 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-05-05 15:55:26,536 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-05-05 15:55:26,537 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,537 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,538 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:26,538 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:26,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:26,538 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:26,538 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:26,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:26,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:26,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:26,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,538 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,538 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:55:26,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:26,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,538 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,539 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:26,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,539 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:26,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,539 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,539 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:26,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,539 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,539 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,539 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,539 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-05 15:55:26,539 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-05 15:55:26,540 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:26,539 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:26,540 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-05 15:55:26,540 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-05 15:55:26,540 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-05 15:55:26,540 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-05 15:55:26,540 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,540 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-05 15:55:26,540 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:55:26,540 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. 2023-05-05 15:55:26,540 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:55:26,540 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:26,540 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. started... 2023-05-05 15:55:26,540 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-05 15:55:26,540 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-05 15:55:26,540 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-05 15:55:26,541 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 94b959e0a954a0b5ed4ceaeaa080b44c 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-05 15:55:26,542 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,542 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,542 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:26,543 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,543 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,543 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:26,552 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf1/f1739bfafb3645d5a7cc0983a90af737 2023-05-05 15:55:26,566 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf2/8c9ed392fcd34fd59ff49859c3e4924f 2023-05-05 15:55:26,569 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf1/f1739bfafb3645d5a7cc0983a90af737 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/f1739bfafb3645d5a7cc0983a90af737 2023-05-05 15:55:26,572 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/f1739bfafb3645d5a7cc0983a90af737, entries=2, sequenceid=6, filesize=4.8 K 2023-05-05 15:55:26,573 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf2/8c9ed392fcd34fd59ff49859c3e4924f as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/8c9ed392fcd34fd59ff49859c3e4924f 2023-05-05 15:55:26,576 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/8c9ed392fcd34fd59ff49859c3e4924f, entries=2, sequenceid=6, filesize=4.8 K 2023-05-05 15:55:26,577 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 94b959e0a954a0b5ed4ceaeaa080b44c in 36ms, sequenceid=6, compaction requested=false 2023-05-05 15:55:26,577 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-05-05 15:55:26,577 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 94b959e0a954a0b5ed4ceaeaa080b44c: 2023-05-05 15:55:26,577 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. 2023-05-05 15:55:26,577 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:55:26,577 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:55:26,577 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,577 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-05 15:55:26,578 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-05 15:55:26,580 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,580 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:26,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:26,580 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,580 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:26,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:26,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:26,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:26,582 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-05-05 15:55:26,582 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3021a3c2[Count = 0] remaining members to acquire global barrier 2023-05-05 15:55:26,582 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-05-05 15:55:26,582 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,583 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,583 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,583 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,583 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-05 15:55:26,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,583 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,583 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-05 15:55:26,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,583 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-05 15:55:26,584 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-05 15:55:26,584 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-05 15:55:26,584 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:55:26,584 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-05 15:55:26,583 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:55:26,584 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:55:26,584 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:55:26,585 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-05 15:55:26,585 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:26,585 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-05 15:55:26,586 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,586 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-05 15:55:26,586 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-05 15:55:26,586 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:26,586 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:26,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:26,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:26,586 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-05 15:55:26,586 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-05 15:55:26,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:26,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:26,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:26,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:55:26,589 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-05-05 15:55:26,589 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-05-05 15:55:26,589 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:55:26,589 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:55:26,589 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,589 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:55:26,591 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,591 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,591 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,591 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:26,591 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:26,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,591 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:26,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:26,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,592 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,591 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:26,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:26,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:26,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:26,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:26,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:26,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:26,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,592 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:26,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,593 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:26,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,594 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:26,594 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,595 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,600 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,600 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,600 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:26,600 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:26,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:26,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:26,600 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:26,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:26,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:26,600 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-05-05 15:55:26,600 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:55:26,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:26,600 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:26,600 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:26,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:26,600 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:26,600 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:26,601 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-05-05 15:55:26,600 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:26,601 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,601 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:55:26,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:26,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:26,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:26,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:26,601 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:26,601 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,601 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,601 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,601 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,601 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,601 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,701 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:55:26,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:55:26,712 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:55:26,714 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-05-05 15:55:26,714 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,714 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-05-05 15:55:26,714 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:26,714 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-05-05 15:55:26,714 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-05-05 15:55:26,714 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,714 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,715 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:26,715 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:26,715 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:26,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:26,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:26,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:26,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:26,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:26,715 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:26,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,716 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,716 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:55:26,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,716 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,716 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:26,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,717 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,717 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:26,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,717 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,717 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-05 15:55:26,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:26,717 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-05 15:55:26,717 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,717 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,717 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-05 15:55:26,717 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-05 15:55:26,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,717 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:26,717 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:26,717 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-05 15:55:26,717 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,717 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-05 15:55:26,717 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-05 15:55:26,717 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-05 15:55:26,717 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-05 15:55:26,718 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,718 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-05 15:55:26,718 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:55:26,718 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:55:26,718 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:26,718 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. 2023-05-05 15:55:26,718 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-05 15:55:26,718 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. started... 2023-05-05 15:55:26,718 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-05 15:55:26,718 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-05 15:55:26,718 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 94b959e0a954a0b5ed4ceaeaa080b44c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:55:26,719 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,720 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,720 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,720 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:26,721 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,721 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:26,730 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf1/a3ba11453c954508a5e695299bc1994c 2023-05-05 15:55:26,740 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf2/1eddb2bbca224552bf6f36c75678a48f 2023-05-05 15:55:26,743 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf1/a3ba11453c954508a5e695299bc1994c as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/a3ba11453c954508a5e695299bc1994c 2023-05-05 15:55:26,747 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/a3ba11453c954508a5e695299bc1994c, entries=10, sequenceid=19, filesize=5.0 K 2023-05-05 15:55:26,747 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf2/1eddb2bbca224552bf6f36c75678a48f as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/1eddb2bbca224552bf6f36c75678a48f 2023-05-05 15:55:26,750 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/1eddb2bbca224552bf6f36c75678a48f, entries=10, sequenceid=19, filesize=5.0 K 2023-05-05 15:55:26,751 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 94b959e0a954a0b5ed4ceaeaa080b44c in 33ms, sequenceid=19, compaction requested=false 2023-05-05 15:55:26,751 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 94b959e0a954a0b5ed4ceaeaa080b44c: 2023-05-05 15:55:26,751 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. 2023-05-05 15:55:26,751 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:55:26,751 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:55:26,751 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,751 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-05 15:55:26,751 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-05 15:55:26,753 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:26,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:26,753 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:26,753 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,753 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:26,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:26,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:26,755 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-05-05 15:55:26,755 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@13b3062e[Count = 0] remaining members to acquire global barrier 2023-05-05 15:55:26,755 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-05-05 15:55:26,755 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,756 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,756 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,756 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,756 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,757 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:55:26,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,756 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-05 15:55:26,757 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-05 15:55:26,757 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-05 15:55:26,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,757 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:55:26,757 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-05 15:55:26,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,757 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:55:26,757 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-05 15:55:26,757 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-05 15:55:26,757 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:55:26,758 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,758 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-05 15:55:26,758 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:26,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:26,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:26,758 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-05 15:55:26,760 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-05 15:55:26,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:26,760 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-05 15:55:26,760 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:26,760 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:26,760 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-05 15:55:26,760 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-05 15:55:26,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:26,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:26,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:55:26,762 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-05-05 15:55:26,762 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-05-05 15:55:26,762 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:55:26,762 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:55:26,762 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,762 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:55:26,764 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,764 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,764 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:26,764 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,764 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:26,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:26,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:26,764 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:26,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:26,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,765 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:26,764 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:26,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:26,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:26,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:26,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:26,766 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,766 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:26,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,767 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:26,768 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,768 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,770 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,770 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:26,770 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:26,770 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:26,770 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:26,770 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:26,770 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-05-05 15:55:26,771 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:55:26,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:26,770 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:26,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:26,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:26,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:26,770 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:26,771 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,771 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,770 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:26,771 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:26,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:26,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:26,771 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-05-05 15:55:26,771 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:26,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:26,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:26,771 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,771 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:55:26,771 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:26,772 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,772 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,772 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,872 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:55:26,873 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:55:26,883 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:55:26,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-05-05 15:55:26,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,885 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-05-05 15:55:26,885 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:26,885 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-05-05 15:55:26,885 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-05-05 15:55:26,885 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,885 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,886 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:26,887 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:26,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:26,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:26,886 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,886 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:26,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:26,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:26,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:26,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:26,887 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,887 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:55:26,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,887 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:26,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,888 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:26,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,888 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:26,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,888 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-05 15:55:26,888 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,888 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,888 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-05 15:55:26,888 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-05 15:55:26,889 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:26,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,889 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-05 15:55:26,889 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-05 15:55:26,889 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:26,889 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-05 15:55:26,889 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-05 15:55:26,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,889 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-05 15:55:26,889 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-05 15:55:26,889 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:55:26,889 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-05 15:55:26,889 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:26,889 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. 2023-05-05 15:55:26,889 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:55:26,889 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. started... 2023-05-05 15:55:26,889 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-05 15:55:26,889 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-05 15:55:26,889 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-05 15:55:26,889 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 94b959e0a954a0b5ed4ceaeaa080b44c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:55:26,891 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,892 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,892 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,892 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:26,893 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,893 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:26,903 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf1/acde3b42e0a24b669d9de481e2ed7a58 2023-05-05 15:55:26,916 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf2/ce273ec523884f0bb9bbbaae1101bf9a 2023-05-05 15:55:26,920 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf1/acde3b42e0a24b669d9de481e2ed7a58 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/acde3b42e0a24b669d9de481e2ed7a58 2023-05-05 15:55:26,923 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/acde3b42e0a24b669d9de481e2ed7a58, entries=10, sequenceid=32, filesize=5.0 K 2023-05-05 15:55:26,924 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf2/ce273ec523884f0bb9bbbaae1101bf9a as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/ce273ec523884f0bb9bbbaae1101bf9a 2023-05-05 15:55:26,927 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/ce273ec523884f0bb9bbbaae1101bf9a, entries=10, sequenceid=32, filesize=5.0 K 2023-05-05 15:55:26,929 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 94b959e0a954a0b5ed4ceaeaa080b44c in 40ms, sequenceid=32, compaction requested=true 2023-05-05 15:55:26,929 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 94b959e0a954a0b5ed4ceaeaa080b44c: 2023-05-05 15:55:26,929 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. 2023-05-05 15:55:26,929 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:55:26,929 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:55:26,929 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,929 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-05 15:55:26,929 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-05 15:55:26,930 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,930 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:26,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:26,931 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,931 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:26,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:26,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:26,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:26,933 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-05-05 15:55:26,933 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@21fdaed1[Count = 0] remaining members to acquire global barrier 2023-05-05 15:55:26,933 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-05-05 15:55:26,933 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,934 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,934 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,934 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,934 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-05 15:55:26,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,934 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,934 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-05 15:55:26,934 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-05 15:55:26,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,934 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:55:26,934 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-05 15:55:26,934 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-05 15:55:26,934 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-05 15:55:26,934 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:55:26,934 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:55:26,934 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:55:26,940 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-05 15:55:26,940 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:26,940 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-05 15:55:26,941 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-05 15:55:26,941 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-05 15:55:26,941 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:26,941 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,941 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-05 15:55:26,941 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:26,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:26,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:26,941 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-05 15:55:26,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:26,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:26,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:26,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:55:26,944 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-05-05 15:55:26,944 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-05-05 15:55:26,944 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:55:26,944 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:55:26,944 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,944 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:55:26,946 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,946 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,946 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,946 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:26,946 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,946 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:26,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:26,946 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,946 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:26,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:26,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:26,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:26,946 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:26,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:26,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:26,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:26,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:26,947 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:26,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,948 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,949 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,949 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:26,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,952 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,952 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:26,952 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:26,952 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:26,952 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:26,952 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,952 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:26,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:26,952 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-05-05 15:55:26,952 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:55:26,952 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:26,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:26,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:26,952 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:26,952 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,953 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,953 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:26,953 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,953 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-05-05 15:55:26,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:26,953 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:55:26,953 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:26,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:26,953 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:26,953 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,953 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:26,954 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:26,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:26,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:26,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:26,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:26,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:26,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:27,053 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:55:27,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:55:27,064 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:55:27,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-05-05 15:55:27,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,065 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-05-05 15:55:27,065 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:27,066 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-05-05 15:55:27,066 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-05-05 15:55:27,066 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,066 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,068 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,068 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:27,068 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:27,068 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:27,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:27,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:27,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:27,068 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:27,068 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:55:27,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:27,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:27,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,069 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,069 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,069 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:27,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,069 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:27,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:27,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,070 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-05 15:55:27,069 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,070 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,070 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,070 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-05 15:55:27,070 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:27,070 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-05 15:55:27,070 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-05 15:55:27,070 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-05 15:55:27,070 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,070 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-05 15:55:27,070 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-05 15:55:27,070 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-05 15:55:27,070 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,070 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:27,070 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-05 15:55:27,070 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:27,070 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-05 15:55:27,070 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-05 15:55:27,070 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-05 15:55:27,070 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-05 15:55:27,070 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:55:27,070 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:55:27,070 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. 2023-05-05 15:55:27,070 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. started... 2023-05-05 15:55:27,071 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 94b959e0a954a0b5ed4ceaeaa080b44c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:55:27,071 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,072 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,072 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,072 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:27,073 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,073 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:27,081 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf1/7d586b2a36f847d88aa92288ef95a741 2023-05-05 15:55:27,091 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf2/a60e6b5fcbe54c429c7d4ea740445f73 2023-05-05 15:55:27,095 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf1/7d586b2a36f847d88aa92288ef95a741 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/7d586b2a36f847d88aa92288ef95a741 2023-05-05 15:55:27,098 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/7d586b2a36f847d88aa92288ef95a741, entries=10, sequenceid=45, filesize=5.0 K 2023-05-05 15:55:27,099 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf2/a60e6b5fcbe54c429c7d4ea740445f73 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/a60e6b5fcbe54c429c7d4ea740445f73 2023-05-05 15:55:27,101 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/a60e6b5fcbe54c429c7d4ea740445f73, entries=10, sequenceid=45, filesize=5.0 K 2023-05-05 15:55:27,102 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 94b959e0a954a0b5ed4ceaeaa080b44c in 31ms, sequenceid=45, compaction requested=true 2023-05-05 15:55:27,102 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 94b959e0a954a0b5ed4ceaeaa080b44c: 2023-05-05 15:55:27,102 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. 2023-05-05 15:55:27,102 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:55:27,103 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:55:27,103 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,103 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-05 15:55:27,103 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-05 15:55:27,104 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,104 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:27,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:27,105 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,105 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:27,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:27,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:27,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:27,106 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-05-05 15:55:27,106 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4674a81a[Count = 0] remaining members to acquire global barrier 2023-05-05 15:55:27,106 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-05-05 15:55:27,106 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,108 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,108 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,108 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,109 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,109 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-05 15:55:27,109 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:55:27,109 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-05 15:55:27,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,109 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-05 15:55:27,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,109 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-05 15:55:27,109 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:55:27,109 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-05 15:55:27,109 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-05 15:55:27,109 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:55:27,109 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:55:27,110 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-05 15:55:27,110 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,110 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:27,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:27,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:27,110 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-05 15:55:27,110 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:27,110 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-05 15:55:27,110 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:27,110 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-05 15:55:27,111 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-05 15:55:27,110 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-05 15:55:27,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:27,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:27,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:27,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:55:27,113 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-05-05 15:55:27,113 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-05-05 15:55:27,113 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:55:27,113 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:55:27,113 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,113 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:55:27,115 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,115 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,115 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,115 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:27,115 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,115 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:27,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:27,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:27,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,115 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:27,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:27,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:27,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:27,115 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:27,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:27,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:27,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:27,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,116 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:27,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,117 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:27,118 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,118 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,119 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,123 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,123 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,123 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,123 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,123 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:27,123 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:27,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:27,124 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-05-05 15:55:27,124 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:55:27,124 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,123 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:27,124 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:27,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:27,124 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,124 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-05-05 15:55:27,124 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:27,124 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:27,124 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:27,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:27,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:27,124 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:27,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:27,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:27,124 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:27,124 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:55:27,124 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:27,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:27,125 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,125 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:27,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:27,225 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:55:27,225 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:55:27,236 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:55:27,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-05-05 15:55:27,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,238 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-05-05 15:55:27,238 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:27,238 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-05-05 15:55:27,238 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-05-05 15:55:27,238 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,238 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,239 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:27,239 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:27,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:27,239 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:27,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:27,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:27,239 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:27,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:27,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:27,239 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,240 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:55:27,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,240 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,240 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:27,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,240 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:27,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,240 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,240 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:27,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,241 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,241 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-05 15:55:27,241 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-05 15:55:27,241 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,241 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-05 15:55:27,241 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-05 15:55:27,241 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,241 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:27,241 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-05 15:55:27,241 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-05 15:55:27,241 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:27,241 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-05 15:55:27,241 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-05 15:55:27,241 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-05 15:55:27,241 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,241 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-05 15:55:27,241 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:27,241 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. 2023-05-05 15:55:27,241 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:55:27,241 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. started... 2023-05-05 15:55:27,241 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-05 15:55:27,241 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:55:27,241 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-05 15:55:27,241 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-05 15:55:27,241 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 94b959e0a954a0b5ed4ceaeaa080b44c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:55:27,243 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,244 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,244 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,244 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:27,244 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,244 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:27,250 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf1/ed160bbb97b0443795d0c30775736dc0 2023-05-05 15:55:27,260 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf2/9cf9dcbd6a8b4f7dafa8a62eb3b45f0b 2023-05-05 15:55:27,263 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf1/ed160bbb97b0443795d0c30775736dc0 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/ed160bbb97b0443795d0c30775736dc0 2023-05-05 15:55:27,266 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/ed160bbb97b0443795d0c30775736dc0, entries=10, sequenceid=58, filesize=5.0 K 2023-05-05 15:55:27,267 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf2/9cf9dcbd6a8b4f7dafa8a62eb3b45f0b as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/9cf9dcbd6a8b4f7dafa8a62eb3b45f0b 2023-05-05 15:55:27,270 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/9cf9dcbd6a8b4f7dafa8a62eb3b45f0b, entries=10, sequenceid=58, filesize=5.0 K 2023-05-05 15:55:27,270 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 94b959e0a954a0b5ed4ceaeaa080b44c in 29ms, sequenceid=58, compaction requested=true 2023-05-05 15:55:27,271 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 94b959e0a954a0b5ed4ceaeaa080b44c: 2023-05-05 15:55:27,271 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. 2023-05-05 15:55:27,271 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:55:27,271 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:55:27,271 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,271 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-05 15:55:27,271 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-05 15:55:27,272 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,272 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:27,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:27,272 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:27,272 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:27,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:27,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:27,274 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-05-05 15:55:27,274 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7bf62ca5[Count = 0] remaining members to acquire global barrier 2023-05-05 15:55:27,274 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-05-05 15:55:27,274 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,275 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,275 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,275 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,275 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,275 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:55:27,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,275 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-05 15:55:27,275 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-05 15:55:27,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,275 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-05 15:55:27,276 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-05 15:55:27,276 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:55:27,275 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-05 15:55:27,276 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:55:27,276 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-05 15:55:27,276 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:55:27,278 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-05 15:55:27,278 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:27,278 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-05 15:55:27,278 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,278 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-05 15:55:27,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:27,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:27,278 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-05 15:55:27,278 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:27,278 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:27,279 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-05 15:55:27,279 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-05 15:55:27,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:27,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:27,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:27,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:55:27,282 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-05-05 15:55:27,282 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-05-05 15:55:27,282 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:55:27,282 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:55:27,282 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,282 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:55:27,283 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,283 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,283 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,283 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:27,283 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,283 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:27,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,284 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:27,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:27,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:27,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:27,284 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:27,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:27,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:27,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:27,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:27,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:27,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,285 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,285 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:27,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,286 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:27,286 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,287 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,289 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:27,289 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:27,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:27,289 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:27,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:27,289 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,289 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:27,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:27,289 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:27,289 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:27,289 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-05-05 15:55:27,290 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:55:27,289 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:27,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:27,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:27,289 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,290 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:27,290 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-05-05 15:55:27,290 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:55:27,289 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:27,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:27,290 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:27,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:27,290 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,290 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,290 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,290 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,291 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,291 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:27,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:27,390 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:55:27,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:55:27,401 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:55:27,402 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-05-05 15:55:27,402 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,402 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-05-05 15:55:27,402 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:27,402 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-05-05 15:55:27,402 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-05-05 15:55:27,403 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,403 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,405 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:27,405 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:27,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:27,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:27,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:27,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:27,405 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,405 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:27,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:27,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:27,405 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,405 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:55:27,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,406 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,406 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,406 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:27,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:27,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:27,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,406 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,406 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,406 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,406 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-05 15:55:27,406 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,406 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-05 15:55:27,406 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-05 15:55:27,406 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,406 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-05 15:55:27,406 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-05 15:55:27,406 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:27,407 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:27,406 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,407 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-05 15:55:27,407 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-05 15:55:27,407 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:27,407 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-05 15:55:27,407 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-05 15:55:27,407 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:55:27,407 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:55:27,407 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. 2023-05-05 15:55:27,407 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-05 15:55:27,407 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-05 15:55:27,407 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-05 15:55:27,407 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. started... 2023-05-05 15:55:27,407 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-05 15:55:27,407 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 94b959e0a954a0b5ed4ceaeaa080b44c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:55:27,408 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,409 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,409 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,409 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:27,409 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,409 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:27,416 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf1/1cd6c96e835c4684a2c9deec40a0a469 2023-05-05 15:55:27,426 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf2/8360124304a6402bac5e11aca7902e99 2023-05-05 15:55:27,429 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf1/1cd6c96e835c4684a2c9deec40a0a469 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/1cd6c96e835c4684a2c9deec40a0a469 2023-05-05 15:55:27,433 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/1cd6c96e835c4684a2c9deec40a0a469, entries=10, sequenceid=71, filesize=5.0 K 2023-05-05 15:55:27,433 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf2/8360124304a6402bac5e11aca7902e99 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/8360124304a6402bac5e11aca7902e99 2023-05-05 15:55:27,436 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/8360124304a6402bac5e11aca7902e99, entries=10, sequenceid=71, filesize=5.0 K 2023-05-05 15:55:27,437 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 94b959e0a954a0b5ed4ceaeaa080b44c in 30ms, sequenceid=71, compaction requested=true 2023-05-05 15:55:27,437 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 94b959e0a954a0b5ed4ceaeaa080b44c: 2023-05-05 15:55:27,437 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. 2023-05-05 15:55:27,437 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:55:27,437 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:55:27,437 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,437 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-05 15:55:27,437 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-05 15:55:27,439 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,439 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:27,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:27,439 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,439 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:27,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:27,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:27,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:27,441 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-05-05 15:55:27,441 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@70ffc783[Count = 0] remaining members to acquire global barrier 2023-05-05 15:55:27,441 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-05-05 15:55:27,441 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,445 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,445 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,445 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-05 15:55:27,445 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-05 15:55:27,446 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-05 15:55:27,446 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:55:27,445 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,445 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-05 15:55:27,446 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:55:27,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,446 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,446 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-05 15:55:27,446 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:55:27,446 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-05 15:55:27,446 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:55:27,447 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,447 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-05 15:55:27,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:27,447 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:27,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:27,447 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-05 15:55:27,447 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-05 15:55:27,447 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:27,447 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-05 15:55:27,448 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-05 15:55:27,448 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:27,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:27,448 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-05 15:55:27,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:27,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:27,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:55:27,451 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-05-05 15:55:27,451 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-05-05 15:55:27,451 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:55:27,451 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:55:27,451 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,451 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:55:27,453 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,453 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,453 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:27,453 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,453 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,453 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:27,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:27,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:27,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,453 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,453 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:27,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:27,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:27,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:27,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:27,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:27,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:27,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:27,454 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,454 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:27,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:27,456 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,456 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,456 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,459 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:27,459 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:27,459 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:27,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:27,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:27,459 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:27,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:27,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:27,459 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:27,459 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-05-05 15:55:27,459 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:55:27,459 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:27,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:27,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:27,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:27,459 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,459 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:27,459 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:27,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:27,459 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,459 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-05-05 15:55:27,459 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:27,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:27,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:27,460 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,460 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:55:27,460 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:27,460 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,460 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,460 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:27,460 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,460 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-05 15:55:27,560 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:55:27,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:55:27,569 DEBUG [Listener at localhost/35117] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,569 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=94b959e0a954a0b5ed4ceaeaa080b44c, daughterA=319ab08408008171bab242968a65d339, daughterB=818a1268dceef7f73d6ac0ae10bbd67e 2023-05-05 15:55:27,571 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=94b959e0a954a0b5ed4ceaeaa080b44c, daughterA=319ab08408008171bab242968a65d339, daughterB=818a1268dceef7f73d6ac0ae10bbd67e 2023-05-05 15:55:27,571 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=94b959e0a954a0b5ed4ceaeaa080b44c, daughterA=319ab08408008171bab242968a65d339, daughterB=818a1268dceef7f73d6ac0ae10bbd67e 2023-05-05 15:55:27,571 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=94b959e0a954a0b5ed4ceaeaa080b44c, daughterA=319ab08408008171bab242968a65d339, daughterB=818a1268dceef7f73d6ac0ae10bbd67e 2023-05-05 15:55:27,572 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=94b959e0a954a0b5ed4ceaeaa080b44c, UNASSIGN}] 2023-05-05 15:55:27,573 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=94b959e0a954a0b5ed4ceaeaa080b44c, UNASSIGN 2023-05-05 15:55:27,573 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=94b959e0a954a0b5ed4ceaeaa080b44c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:27,573 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683302127573"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302127573"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302127573"}]},"ts":"1683302127573"} 2023-05-05 15:55:27,574 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 94b959e0a954a0b5ed4ceaeaa080b44c, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:55:27,726 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,726 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 94b959e0a954a0b5ed4ceaeaa080b44c, disabling compactions & flushes 2023-05-05 15:55:27,726 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. 2023-05-05 15:55:27,726 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. 2023-05-05 15:55:27,726 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. after waiting 0 ms 2023-05-05 15:55:27,726 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. 2023-05-05 15:55:27,726 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 94b959e0a954a0b5ed4ceaeaa080b44c 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-05 15:55:27,735 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf1/7ffe6e4dbe294595b1344cf1d13278fd 2023-05-05 15:55:27,745 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf2/08fd52360b9f4a17b4780fd97344db08 2023-05-05 15:55:27,748 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf1/7ffe6e4dbe294595b1344cf1d13278fd as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/7ffe6e4dbe294595b1344cf1d13278fd 2023-05-05 15:55:27,751 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/7ffe6e4dbe294595b1344cf1d13278fd, entries=8, sequenceid=82, filesize=5.0 K 2023-05-05 15:55:27,751 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.tmp/cf2/08fd52360b9f4a17b4780fd97344db08 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/08fd52360b9f4a17b4780fd97344db08 2023-05-05 15:55:27,754 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/08fd52360b9f4a17b4780fd97344db08, entries=8, sequenceid=82, filesize=5.0 K 2023-05-05 15:55:27,755 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 94b959e0a954a0b5ed4ceaeaa080b44c in 29ms, sequenceid=82, compaction requested=true 2023-05-05 15:55:27,760 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-05 15:55:27,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:55:27,761 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. 2023-05-05 15:55:27,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 94b959e0a954a0b5ed4ceaeaa080b44c: 2023-05-05 15:55:27,762 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,763 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=94b959e0a954a0b5ed4ceaeaa080b44c, regionState=CLOSED 2023-05-05 15:55:27,763 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683302127763"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302127763"}]},"ts":"1683302127763"} 2023-05-05 15:55:27,765 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-05-05 15:55:27,765 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 94b959e0a954a0b5ed4ceaeaa080b44c, server=jenkins-hbase4.apache.org,40991,1683302023016 in 190 msec 2023-05-05 15:55:27,767 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-05-05 15:55:27,767 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=94b959e0a954a0b5ed4ceaeaa080b44c, UNASSIGN in 193 msec 2023-05-05 15:55:27,776 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=94b959e0a954a0b5ed4ceaeaa080b44c, threads=14 2023-05-05 15:55:27,776 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/08fd52360b9f4a17b4780fd97344db08 for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,776 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/1eddb2bbca224552bf6f36c75678a48f for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,776 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/8360124304a6402bac5e11aca7902e99 for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,776 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/8c9ed392fcd34fd59ff49859c3e4924f for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,776 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/9cf9dcbd6a8b4f7dafa8a62eb3b45f0b for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,776 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/a60e6b5fcbe54c429c7d4ea740445f73 for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,776 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/ce273ec523884f0bb9bbbaae1101bf9a for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,777 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/1cd6c96e835c4684a2c9deec40a0a469 for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,777 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/7d586b2a36f847d88aa92288ef95a741 for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,777 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/7ffe6e4dbe294595b1344cf1d13278fd for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,777 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/a3ba11453c954508a5e695299bc1994c for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,777 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/acde3b42e0a24b669d9de481e2ed7a58 for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,777 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/ed160bbb97b0443795d0c30775736dc0 for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,777 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/f1739bfafb3645d5a7cc0983a90af737 for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,780 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/9cf9dcbd6a8b4f7dafa8a62eb3b45f0b, top=false 2023-05-05 15:55:27,781 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/1eddb2bbca224552bf6f36c75678a48f, top=false 2023-05-05 15:55:27,782 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/08fd52360b9f4a17b4780fd97344db08, top=false 2023-05-05 15:55:27,782 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/7ffe6e4dbe294595b1344cf1d13278fd, top=false 2023-05-05 15:55:27,794 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/a60e6b5fcbe54c429c7d4ea740445f73, top=false 2023-05-05 15:55:27,795 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/f1739bfafb3645d5a7cc0983a90af737, top=false 2023-05-05 15:55:27,796 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/ce273ec523884f0bb9bbbaae1101bf9a, top=false 2023-05-05 15:55:27,797 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/1cd6c96e835c4684a2c9deec40a0a469, top=false 2023-05-05 15:55:27,797 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/8c9ed392fcd34fd59ff49859c3e4924f, top=false 2023-05-05 15:55:27,797 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/7d586b2a36f847d88aa92288ef95a741, top=false 2023-05-05 15:55:27,800 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/ed160bbb97b0443795d0c30775736dc0, top=false 2023-05-05 15:55:27,801 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/acde3b42e0a24b669d9de481e2ed7a58, top=false 2023-05-05 15:55:27,801 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/8360124304a6402bac5e11aca7902e99, top=false 2023-05-05 15:55:27,806 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/a3ba11453c954508a5e695299bc1994c, top=false 2023-05-05 15:55:27,808 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.splits/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-9cf9dcbd6a8b4f7dafa8a62eb3b45f0b for child: 319ab08408008171bab242968a65d339, parent: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,811 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/9cf9dcbd6a8b4f7dafa8a62eb3b45f0b for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,816 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.splits/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-7ffe6e4dbe294595b1344cf1d13278fd for child: 319ab08408008171bab242968a65d339, parent: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,816 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.splits/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-08fd52360b9f4a17b4780fd97344db08 for child: 319ab08408008171bab242968a65d339, parent: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,821 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/7ffe6e4dbe294595b1344cf1d13278fd for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,821 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/08fd52360b9f4a17b4780fd97344db08 for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,832 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.splits/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-f1739bfafb3645d5a7cc0983a90af737 for child: 319ab08408008171bab242968a65d339, parent: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,832 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.splits/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-1eddb2bbca224552bf6f36c75678a48f for child: 319ab08408008171bab242968a65d339, parent: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,832 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.splits/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-a60e6b5fcbe54c429c7d4ea740445f73 for child: 319ab08408008171bab242968a65d339, parent: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,842 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.splits/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-a3ba11453c954508a5e695299bc1994c for child: 319ab08408008171bab242968a65d339, parent: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,842 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.splits/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-ce273ec523884f0bb9bbbaae1101bf9a for child: 319ab08408008171bab242968a65d339, parent: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,842 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.splits/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-1cd6c96e835c4684a2c9deec40a0a469 for child: 319ab08408008171bab242968a65d339, parent: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,843 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.splits/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-8c9ed392fcd34fd59ff49859c3e4924f for child: 319ab08408008171bab242968a65d339, parent: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,843 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.splits/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-7d586b2a36f847d88aa92288ef95a741 for child: 319ab08408008171bab242968a65d339, parent: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,844 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/1eddb2bbca224552bf6f36c75678a48f for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,844 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/a60e6b5fcbe54c429c7d4ea740445f73 for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,844 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.splits/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-ed160bbb97b0443795d0c30775736dc0 for child: 319ab08408008171bab242968a65d339, parent: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,844 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/f1739bfafb3645d5a7cc0983a90af737 for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,846 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.splits/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-8360124304a6402bac5e11aca7902e99 for child: 319ab08408008171bab242968a65d339, parent: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,847 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/.splits/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-acde3b42e0a24b669d9de481e2ed7a58 for child: 319ab08408008171bab242968a65d339, parent: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,849 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/7d586b2a36f847d88aa92288ef95a741 for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,849 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/8c9ed392fcd34fd59ff49859c3e4924f for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,850 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/a3ba11453c954508a5e695299bc1994c for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,850 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/ed160bbb97b0443795d0c30775736dc0 for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,850 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/1cd6c96e835c4684a2c9deec40a0a469 for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,851 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/8360124304a6402bac5e11aca7902e99 for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,851 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/acde3b42e0a24b669d9de481e2ed7a58 for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,852 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/ce273ec523884f0bb9bbbaae1101bf9a for region: 94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:27,852 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 94b959e0a954a0b5ed4ceaeaa080b44c Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-05-05 15:55:27,888 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-05 15:55:27,890 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/818a1268dceef7f73d6ac0ae10bbd67e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-05 15:55:27,892 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683302127892"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1683302127892"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1683302127892"}]},"ts":"1683302127892"} 2023-05-05 15:55:27,892 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683302127892"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302127892"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302127892"}]},"ts":"1683302127892"} 2023-05-05 15:55:27,892 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1683302127569.818a1268dceef7f73d6ac0ae10bbd67e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683302127892"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302127892"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302127892"}]},"ts":"1683302127892"} 2023-05-05 15:55:27,895 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=319ab08408008171bab242968a65d339, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=818a1268dceef7f73d6ac0ae10bbd67e, ASSIGN}] 2023-05-05 15:55:27,895 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=818a1268dceef7f73d6ac0ae10bbd67e, ASSIGN 2023-05-05 15:55:27,896 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=818a1268dceef7f73d6ac0ae10bbd67e, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40991,1683302023016; forceNewPlan=false, retain=false 2023-05-05 15:55:27,897 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=319ab08408008171bab242968a65d339, ASSIGN 2023-05-05 15:55:27,897 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=319ab08408008171bab242968a65d339, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40991,1683302023016; forceNewPlan=false, retain=false 2023-05-05 15:55:27,947 ERROR [BP-1744589926-172.31.14.131-1683302017433 heartbeating to localhost/127.0.0.1:35855] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1744589926-172.31.14.131-1683302017433 (Datanode Uuid 8e12ece6-ff16-4724-9684-7233909ecb4c) service to localhost/127.0.0.1:35855 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-05-05 15:55:28,015 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-05 15:55:28,046 INFO [jenkins-hbase4:44213] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-05 15:55:28,049 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=818a1268dceef7f73d6ac0ae10bbd67e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:28,049 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1683302127569.818a1268dceef7f73d6ac0ae10bbd67e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683302128048"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302128048"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302128048"}]},"ts":"1683302128048"} 2023-05-05 15:55:28,050 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=132, state=RUNNABLE; OpenRegionProcedure 818a1268dceef7f73d6ac0ae10bbd67e, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:55:28,053 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=319ab08408008171bab242968a65d339, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:28,053 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683302128053"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302128053"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302128053"}]},"ts":"1683302128053"} 2023-05-05 15:55:28,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; OpenRegionProcedure 319ab08408008171bab242968a65d339, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:55:28,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,71,1683302127569.818a1268dceef7f73d6ac0ae10bbd67e. 2023-05-05 15:55:28,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 818a1268dceef7f73d6ac0ae10bbd67e, NAME => 'testSplitTableRegionEmptyDaughter,71,1683302127569.818a1268dceef7f73d6ac0ae10bbd67e.', STARTKEY => '71', ENDKEY => ''} 2023-05-05 15:55:28,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:55:28,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 818a1268dceef7f73d6ac0ae10bbd67e 2023-05-05 15:55:28,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,71,1683302127569.818a1268dceef7f73d6ac0ae10bbd67e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:55:28,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 818a1268dceef7f73d6ac0ae10bbd67e 2023-05-05 15:55:28,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 818a1268dceef7f73d6ac0ae10bbd67e 2023-05-05 15:55:28,206 INFO [StoreOpener-818a1268dceef7f73d6ac0ae10bbd67e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 818a1268dceef7f73d6ac0ae10bbd67e 2023-05-05 15:55:28,208 DEBUG [StoreOpener-818a1268dceef7f73d6ac0ae10bbd67e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/818a1268dceef7f73d6ac0ae10bbd67e/cf1 2023-05-05 15:55:28,208 DEBUG [StoreOpener-818a1268dceef7f73d6ac0ae10bbd67e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/818a1268dceef7f73d6ac0ae10bbd67e/cf1 2023-05-05 15:55:28,208 INFO [StoreOpener-818a1268dceef7f73d6ac0ae10bbd67e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 818a1268dceef7f73d6ac0ae10bbd67e columnFamilyName cf1 2023-05-05 15:55:28,208 INFO [StoreOpener-818a1268dceef7f73d6ac0ae10bbd67e-1] regionserver.HStore(310): Store=818a1268dceef7f73d6ac0ae10bbd67e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:55:28,209 INFO [StoreOpener-818a1268dceef7f73d6ac0ae10bbd67e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 818a1268dceef7f73d6ac0ae10bbd67e 2023-05-05 15:55:28,210 DEBUG [StoreOpener-818a1268dceef7f73d6ac0ae10bbd67e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/818a1268dceef7f73d6ac0ae10bbd67e/cf2 2023-05-05 15:55:28,210 DEBUG [StoreOpener-818a1268dceef7f73d6ac0ae10bbd67e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/818a1268dceef7f73d6ac0ae10bbd67e/cf2 2023-05-05 15:55:28,210 INFO [StoreOpener-818a1268dceef7f73d6ac0ae10bbd67e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 818a1268dceef7f73d6ac0ae10bbd67e columnFamilyName cf2 2023-05-05 15:55:28,210 INFO [StoreOpener-818a1268dceef7f73d6ac0ae10bbd67e-1] regionserver.HStore(310): Store=818a1268dceef7f73d6ac0ae10bbd67e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:55:28,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/818a1268dceef7f73d6ac0ae10bbd67e 2023-05-05 15:55:28,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/818a1268dceef7f73d6ac0ae10bbd67e 2023-05-05 15:55:28,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-05 15:55:28,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 818a1268dceef7f73d6ac0ae10bbd67e 2023-05-05 15:55:28,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 818a1268dceef7f73d6ac0ae10bbd67e; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64241465, jitterRate=-0.042727574706077576}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-05 15:55:28,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 818a1268dceef7f73d6ac0ae10bbd67e: 2023-05-05 15:55:28,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1683302127569.818a1268dceef7f73d6ac0ae10bbd67e., pid=133, masterSystemTime=1683302128202 2023-05-05 15:55:28,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1683302127569.818a1268dceef7f73d6ac0ae10bbd67e. 2023-05-05 15:55:28,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,71,1683302127569.818a1268dceef7f73d6ac0ae10bbd67e. 2023-05-05 15:55:28,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339. 2023-05-05 15:55:28,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 319ab08408008171bab242968a65d339, NAME => 'testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.', STARTKEY => '', ENDKEY => '71'} 2023-05-05 15:55:28,216 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=818a1268dceef7f73d6ac0ae10bbd67e, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:28,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:55:28,217 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1683302127569.818a1268dceef7f73d6ac0ae10bbd67e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683302128216"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302128216"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302128216"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302128216"}]},"ts":"1683302128216"} 2023-05-05 15:55:28,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 319ab08408008171bab242968a65d339 2023-05-05 15:55:28,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:55:28,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 319ab08408008171bab242968a65d339 2023-05-05 15:55:28,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 319ab08408008171bab242968a65d339 2023-05-05 15:55:28,218 INFO [StoreOpener-319ab08408008171bab242968a65d339-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 319ab08408008171bab242968a65d339 2023-05-05 15:55:28,219 DEBUG [StoreOpener-319ab08408008171bab242968a65d339-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1 2023-05-05 15:55:28,219 DEBUG [StoreOpener-319ab08408008171bab242968a65d339-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1 2023-05-05 15:55:28,219 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=132 2023-05-05 15:55:28,219 INFO [StoreOpener-319ab08408008171bab242968a65d339-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 319ab08408008171bab242968a65d339 columnFamilyName cf1 2023-05-05 15:55:28,219 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=132, state=SUCCESS; OpenRegionProcedure 818a1268dceef7f73d6ac0ae10bbd67e, server=jenkins-hbase4.apache.org,40991,1683302023016 in 168 msec 2023-05-05 15:55:28,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=818a1268dceef7f73d6ac0ae10bbd67e, ASSIGN in 324 msec 2023-05-05 15:55:28,229 DEBUG [StoreOpener-319ab08408008171bab242968a65d339-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-1cd6c96e835c4684a2c9deec40a0a469 2023-05-05 15:55:28,232 DEBUG [StoreOpener-319ab08408008171bab242968a65d339-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-7d586b2a36f847d88aa92288ef95a741 2023-05-05 15:55:28,235 DEBUG [StoreOpener-319ab08408008171bab242968a65d339-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-7ffe6e4dbe294595b1344cf1d13278fd 2023-05-05 15:55:28,238 DEBUG [StoreOpener-319ab08408008171bab242968a65d339-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-a3ba11453c954508a5e695299bc1994c 2023-05-05 15:55:28,240 DEBUG [StoreOpener-319ab08408008171bab242968a65d339-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-acde3b42e0a24b669d9de481e2ed7a58 2023-05-05 15:55:28,243 DEBUG [StoreOpener-319ab08408008171bab242968a65d339-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-ed160bbb97b0443795d0c30775736dc0 2023-05-05 15:55:28,245 DEBUG [StoreOpener-319ab08408008171bab242968a65d339-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-f1739bfafb3645d5a7cc0983a90af737 2023-05-05 15:55:28,245 INFO [StoreOpener-319ab08408008171bab242968a65d339-1] regionserver.HStore(310): Store=319ab08408008171bab242968a65d339/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:55:28,246 INFO [StoreOpener-319ab08408008171bab242968a65d339-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 319ab08408008171bab242968a65d339 2023-05-05 15:55:28,246 DEBUG [StoreOpener-319ab08408008171bab242968a65d339-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2 2023-05-05 15:55:28,246 DEBUG [StoreOpener-319ab08408008171bab242968a65d339-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2 2023-05-05 15:55:28,247 INFO [StoreOpener-319ab08408008171bab242968a65d339-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 319ab08408008171bab242968a65d339 columnFamilyName cf2 2023-05-05 15:55:28,251 DEBUG [StoreOpener-319ab08408008171bab242968a65d339-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-08fd52360b9f4a17b4780fd97344db08 2023-05-05 15:55:28,253 DEBUG [StoreOpener-319ab08408008171bab242968a65d339-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-1eddb2bbca224552bf6f36c75678a48f 2023-05-05 15:55:28,255 DEBUG [StoreOpener-319ab08408008171bab242968a65d339-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-8360124304a6402bac5e11aca7902e99 2023-05-05 15:55:28,258 DEBUG [StoreOpener-319ab08408008171bab242968a65d339-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-8c9ed392fcd34fd59ff49859c3e4924f 2023-05-05 15:55:28,260 DEBUG [StoreOpener-319ab08408008171bab242968a65d339-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-9cf9dcbd6a8b4f7dafa8a62eb3b45f0b 2023-05-05 15:55:28,263 DEBUG [StoreOpener-319ab08408008171bab242968a65d339-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-a60e6b5fcbe54c429c7d4ea740445f73 2023-05-05 15:55:28,265 DEBUG [StoreOpener-319ab08408008171bab242968a65d339-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-ce273ec523884f0bb9bbbaae1101bf9a 2023-05-05 15:55:28,266 INFO [StoreOpener-319ab08408008171bab242968a65d339-1] regionserver.HStore(310): Store=319ab08408008171bab242968a65d339/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:55:28,266 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339 2023-05-05 15:55:28,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339 2023-05-05 15:55:28,268 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-05 15:55:28,269 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 319ab08408008171bab242968a65d339 2023-05-05 15:55:28,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 319ab08408008171bab242968a65d339; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64174299, jitterRate=-0.04372842609882355}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-05 15:55:28,270 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 319ab08408008171bab242968a65d339: 2023-05-05 15:55:28,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339., pid=134, masterSystemTime=1683302128202 2023-05-05 15:55:28,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:55:28,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-05 15:55:28,272 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-05 15:55:28,272 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-05 15:55:28,272 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-05-05 15:55:28,272 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-05-05 15:55:28,272 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1912): 319ab08408008171bab242968a65d339/cf2 is initiating minor compaction (all files) 2023-05-05 15:55:28,272 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1912): 319ab08408008171bab242968a65d339/cf1 is initiating minor compaction (all files) 2023-05-05 15:55:28,272 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 319ab08408008171bab242968a65d339/cf2 in testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339. 2023-05-05 15:55:28,272 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 319ab08408008171bab242968a65d339/cf1 in testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339. 2023-05-05 15:55:28,272 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-8c9ed392fcd34fd59ff49859c3e4924f, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-1eddb2bbca224552bf6f36c75678a48f, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-ce273ec523884f0bb9bbbaae1101bf9a, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-a60e6b5fcbe54c429c7d4ea740445f73, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-9cf9dcbd6a8b4f7dafa8a62eb3b45f0b, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-8360124304a6402bac5e11aca7902e99, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-08fd52360b9f4a17b4780fd97344db08] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/.tmp, totalSize=34.9 K 2023-05-05 15:55:28,272 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-f1739bfafb3645d5a7cc0983a90af737, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-a3ba11453c954508a5e695299bc1994c, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-acde3b42e0a24b669d9de481e2ed7a58, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-7d586b2a36f847d88aa92288ef95a741, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-ed160bbb97b0443795d0c30775736dc0, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-1cd6c96e835c4684a2c9deec40a0a469, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-7ffe6e4dbe294595b1344cf1d13278fd] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/.tmp, totalSize=34.9 K 2023-05-05 15:55:28,273 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-8c9ed392fcd34fd59ff49859c3e4924f, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683302126530 2023-05-05 15:55:28,273 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-f1739bfafb3645d5a7cc0983a90af737, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683302126530 2023-05-05 15:55:28,273 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-1eddb2bbca224552bf6f36c75678a48f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683302126703 2023-05-05 15:55:28,273 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339. 2023-05-05 15:55:28,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339. 2023-05-05 15:55:28,273 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-a3ba11453c954508a5e695299bc1994c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683302126703 2023-05-05 15:55:28,273 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-ce273ec523884f0bb9bbbaae1101bf9a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683302126873 2023-05-05 15:55:28,273 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-acde3b42e0a24b669d9de481e2ed7a58, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683302126873 2023-05-05 15:55:28,273 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=319ab08408008171bab242968a65d339, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:28,274 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-a60e6b5fcbe54c429c7d4ea740445f73, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683302127055 2023-05-05 15:55:28,274 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683302128273"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302128273"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302128273"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302128273"}]},"ts":"1683302128273"} 2023-05-05 15:55:28,274 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-7d586b2a36f847d88aa92288ef95a741, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683302127055 2023-05-05 15:55:28,274 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-9cf9dcbd6a8b4f7dafa8a62eb3b45f0b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683302127226 2023-05-05 15:55:28,274 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-ed160bbb97b0443795d0c30775736dc0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683302127226 2023-05-05 15:55:28,274 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-8360124304a6402bac5e11aca7902e99, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683302127392 2023-05-05 15:55:28,275 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-1cd6c96e835c4684a2c9deec40a0a469, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683302127392 2023-05-05 15:55:28,275 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-08fd52360b9f4a17b4780fd97344db08, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683302127562 2023-05-05 15:55:28,275 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-7ffe6e4dbe294595b1344cf1d13278fd, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683302127562 2023-05-05 15:55:28,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=131 2023-05-05 15:55:28,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=131, state=SUCCESS; OpenRegionProcedure 319ab08408008171bab242968a65d339, server=jenkins-hbase4.apache.org,40991,1683302023016 in 221 msec 2023-05-05 15:55:28,278 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=131, resume processing ppid=128 2023-05-05 15:55:28,278 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=319ab08408008171bab242968a65d339, ASSIGN in 382 msec 2023-05-05 15:55:28,280 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=94b959e0a954a0b5ed4ceaeaa080b44c, daughterA=319ab08408008171bab242968a65d339, daughterB=818a1268dceef7f73d6ac0ae10bbd67e in 709 msec 2023-05-05 15:55:28,290 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] throttle.PressureAwareThroughputController(145): 319ab08408008171bab242968a65d339#cf2#compaction#126 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:55:28,292 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] throttle.PressureAwareThroughputController(145): 319ab08408008171bab242968a65d339#cf1#compaction#127 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:55:28,308 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/.tmp/cf1/48e4e506cb9349d6b2f0eefcafa31af6 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/48e4e506cb9349d6b2f0eefcafa31af6 2023-05-05 15:55:28,312 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 319ab08408008171bab242968a65d339/cf1 of 319ab08408008171bab242968a65d339 into 48e4e506cb9349d6b2f0eefcafa31af6(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:55:28,312 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 319ab08408008171bab242968a65d339: 2023-05-05 15:55:28,312 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339., storeName=319ab08408008171bab242968a65d339/cf1, priority=9, startTime=1683302128271; duration=0sec 2023-05-05 15:55:28,313 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:55:28,321 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40991] ipc.CallRunner(144): callId: 991 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:60952 deadline: 1683302188320, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c. is not online on jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:28,438 DEBUG [Listener at localhost/35117] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-05-05 15:55:28,441 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-05 15:55:28,442 INFO [Listener at localhost/35117] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-05-05 15:55:28,443 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionEmptyDaughter 2023-05-05 15:55:28,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-05 15:55:28,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-05-05 15:55:28,446 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302128446"}]},"ts":"1683302128446"} 2023-05-05 15:55:28,447 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-05-05 15:55:28,449 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-05-05 15:55:28,449 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=319ab08408008171bab242968a65d339, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=818a1268dceef7f73d6ac0ae10bbd67e, UNASSIGN}] 2023-05-05 15:55:28,450 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=818a1268dceef7f73d6ac0ae10bbd67e, UNASSIGN 2023-05-05 15:55:28,450 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=818a1268dceef7f73d6ac0ae10bbd67e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:28,451 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1683302127569.818a1268dceef7f73d6ac0ae10bbd67e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683302128450"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302128450"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302128450"}]},"ts":"1683302128450"} 2023-05-05 15:55:28,452 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 818a1268dceef7f73d6ac0ae10bbd67e, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:55:28,454 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=319ab08408008171bab242968a65d339, UNASSIGN 2023-05-05 15:55:28,454 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=319ab08408008171bab242968a65d339, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:28,455 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683302128454"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302128454"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302128454"}]},"ts":"1683302128454"} 2023-05-05 15:55:28,456 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure 319ab08408008171bab242968a65d339, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:55:28,547 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-05-05 15:55:28,604 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 818a1268dceef7f73d6ac0ae10bbd67e 2023-05-05 15:55:28,605 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 818a1268dceef7f73d6ac0ae10bbd67e, disabling compactions & flushes 2023-05-05 15:55:28,605 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,71,1683302127569.818a1268dceef7f73d6ac0ae10bbd67e. 2023-05-05 15:55:28,605 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1683302127569.818a1268dceef7f73d6ac0ae10bbd67e. 2023-05-05 15:55:28,605 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1683302127569.818a1268dceef7f73d6ac0ae10bbd67e. after waiting 0 ms 2023-05-05 15:55:28,605 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1683302127569.818a1268dceef7f73d6ac0ae10bbd67e. 2023-05-05 15:55:28,608 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/818a1268dceef7f73d6ac0ae10bbd67e/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-05-05 15:55:28,609 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:55:28,609 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,71,1683302127569.818a1268dceef7f73d6ac0ae10bbd67e. 2023-05-05 15:55:28,609 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 818a1268dceef7f73d6ac0ae10bbd67e: 2023-05-05 15:55:28,610 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 818a1268dceef7f73d6ac0ae10bbd67e 2023-05-05 15:55:28,610 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 319ab08408008171bab242968a65d339 2023-05-05 15:55:28,611 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 319ab08408008171bab242968a65d339, disabling compactions & flushes 2023-05-05 15:55:28,611 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339. 2023-05-05 15:55:28,611 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=818a1268dceef7f73d6ac0ae10bbd67e, regionState=CLOSED 2023-05-05 15:55:28,611 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1683302127569.818a1268dceef7f73d6ac0ae10bbd67e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683302128611"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302128611"}]},"ts":"1683302128611"} 2023-05-05 15:55:28,613 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-05-05 15:55:28,614 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 818a1268dceef7f73d6ac0ae10bbd67e, server=jenkins-hbase4.apache.org,40991,1683302023016 in 160 msec 2023-05-05 15:55:28,614 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=818a1268dceef7f73d6ac0ae10bbd67e, UNASSIGN in 165 msec 2023-05-05 15:55:28,711 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/.tmp/cf2/796b02bf6f8742c682bf9cabcd95db3c as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/796b02bf6f8742c682bf9cabcd95db3c 2023-05-05 15:55:28,714 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 319ab08408008171bab242968a65d339/cf2 of 319ab08408008171bab242968a65d339 into 796b02bf6f8742c682bf9cabcd95db3c(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:55:28,715 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 319ab08408008171bab242968a65d339: 2023-05-05 15:55:28,715 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339. 2023-05-05 15:55:28,715 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339., storeName=319ab08408008171bab242968a65d339/cf2, priority=9, startTime=1683302128272; duration=0sec 2023-05-05 15:55:28,715 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339. 2023-05-05 15:55:28,715 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339. after waiting 0 ms 2023-05-05 15:55:28,715 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:55:28,715 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339. 2023-05-05 15:55:28,715 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-f1739bfafb3645d5a7cc0983a90af737, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-a3ba11453c954508a5e695299bc1994c, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-acde3b42e0a24b669d9de481e2ed7a58, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-7d586b2a36f847d88aa92288ef95a741, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-ed160bbb97b0443795d0c30775736dc0, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-1cd6c96e835c4684a2c9deec40a0a469, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-7ffe6e4dbe294595b1344cf1d13278fd] to archive 2023-05-05 15:55:28,716 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-05 15:55:28,717 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-f1739bfafb3645d5a7cc0983a90af737 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-f1739bfafb3645d5a7cc0983a90af737 2023-05-05 15:55:28,718 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-a3ba11453c954508a5e695299bc1994c to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-a3ba11453c954508a5e695299bc1994c 2023-05-05 15:55:28,719 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-acde3b42e0a24b669d9de481e2ed7a58 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-acde3b42e0a24b669d9de481e2ed7a58 2023-05-05 15:55:28,721 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-7d586b2a36f847d88aa92288ef95a741 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-7d586b2a36f847d88aa92288ef95a741 2023-05-05 15:55:28,722 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-ed160bbb97b0443795d0c30775736dc0 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-ed160bbb97b0443795d0c30775736dc0 2023-05-05 15:55:28,723 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-1cd6c96e835c4684a2c9deec40a0a469 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-1cd6c96e835c4684a2c9deec40a0a469 2023-05-05 15:55:28,724 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-7ffe6e4dbe294595b1344cf1d13278fd to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-7ffe6e4dbe294595b1344cf1d13278fd 2023-05-05 15:55:28,725 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-8c9ed392fcd34fd59ff49859c3e4924f, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-1eddb2bbca224552bf6f36c75678a48f, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-ce273ec523884f0bb9bbbaae1101bf9a, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-a60e6b5fcbe54c429c7d4ea740445f73, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-9cf9dcbd6a8b4f7dafa8a62eb3b45f0b, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-8360124304a6402bac5e11aca7902e99, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-08fd52360b9f4a17b4780fd97344db08] to archive 2023-05-05 15:55:28,726 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-05 15:55:28,727 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-8c9ed392fcd34fd59ff49859c3e4924f to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-8c9ed392fcd34fd59ff49859c3e4924f 2023-05-05 15:55:28,728 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-1eddb2bbca224552bf6f36c75678a48f to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-1eddb2bbca224552bf6f36c75678a48f 2023-05-05 15:55:28,729 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-ce273ec523884f0bb9bbbaae1101bf9a to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-ce273ec523884f0bb9bbbaae1101bf9a 2023-05-05 15:55:28,729 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-a60e6b5fcbe54c429c7d4ea740445f73 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-a60e6b5fcbe54c429c7d4ea740445f73 2023-05-05 15:55:28,730 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-9cf9dcbd6a8b4f7dafa8a62eb3b45f0b to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-9cf9dcbd6a8b4f7dafa8a62eb3b45f0b 2023-05-05 15:55:28,731 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-8360124304a6402bac5e11aca7902e99 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-8360124304a6402bac5e11aca7902e99 2023-05-05 15:55:28,732 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-08fd52360b9f4a17b4780fd97344db08 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/testSplitTableRegionEmptyDaughter=94b959e0a954a0b5ed4ceaeaa080b44c-08fd52360b9f4a17b4780fd97344db08 2023-05-05 15:55:28,735 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-05 15:55:28,735 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:55:28,736 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339. 2023-05-05 15:55:28,736 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 319ab08408008171bab242968a65d339: 2023-05-05 15:55:28,737 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 319ab08408008171bab242968a65d339 2023-05-05 15:55:28,737 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=319ab08408008171bab242968a65d339, regionState=CLOSED 2023-05-05 15:55:28,737 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683302128737"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302128737"}]},"ts":"1683302128737"} 2023-05-05 15:55:28,740 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-05-05 15:55:28,740 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure 319ab08408008171bab242968a65d339, server=jenkins-hbase4.apache.org,40991,1683302023016 in 282 msec 2023-05-05 15:55:28,741 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=136, resume processing ppid=135 2023-05-05 15:55:28,741 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=319ab08408008171bab242968a65d339, UNASSIGN in 291 msec 2023-05-05 15:55:28,742 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302128742"}]},"ts":"1683302128742"} 2023-05-05 15:55:28,743 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-05-05 15:55:28,744 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-05-05 15:55:28,746 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 302 msec 2023-05-05 15:55:28,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-05-05 15:55:28,748 INFO [Listener at localhost/35117] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-05-05 15:55:28,748 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionEmptyDaughter 2023-05-05 15:55:28,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-05 15:55:28,750 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-05 15:55:28,751 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-05-05 15:55:28,751 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-05 15:55:28,755 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:28,755 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/818a1268dceef7f73d6ac0ae10bbd67e 2023-05-05 15:55:28,755 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339 2023-05-05 15:55:28,757 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/818a1268dceef7f73d6ac0ae10bbd67e/cf1, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/818a1268dceef7f73d6ac0ae10bbd67e/cf2, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/818a1268dceef7f73d6ac0ae10bbd67e/recovered.edits] 2023-05-05 15:55:28,758 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/recovered.edits] 2023-05-05 15:55:28,758 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/recovered.edits] 2023-05-05 15:55:28,762 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/48e4e506cb9349d6b2f0eefcafa31af6 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf1/48e4e506cb9349d6b2f0eefcafa31af6 2023-05-05 15:55:28,764 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/1cd6c96e835c4684a2c9deec40a0a469 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/1cd6c96e835c4684a2c9deec40a0a469 2023-05-05 15:55:28,766 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/7d586b2a36f847d88aa92288ef95a741 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/7d586b2a36f847d88aa92288ef95a741 2023-05-05 15:55:28,766 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/796b02bf6f8742c682bf9cabcd95db3c to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/cf2/796b02bf6f8742c682bf9cabcd95db3c 2023-05-05 15:55:28,767 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/818a1268dceef7f73d6ac0ae10bbd67e/recovered.edits/88.seqid to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/818a1268dceef7f73d6ac0ae10bbd67e/recovered.edits/88.seqid 2023-05-05 15:55:28,767 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/818a1268dceef7f73d6ac0ae10bbd67e 2023-05-05 15:55:28,768 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/7ffe6e4dbe294595b1344cf1d13278fd to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/7ffe6e4dbe294595b1344cf1d13278fd 2023-05-05 15:55:28,769 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/a3ba11453c954508a5e695299bc1994c to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/a3ba11453c954508a5e695299bc1994c 2023-05-05 15:55:28,769 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/recovered.edits/90.seqid to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339/recovered.edits/90.seqid 2023-05-05 15:55:28,770 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/319ab08408008171bab242968a65d339 2023-05-05 15:55:28,770 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/acde3b42e0a24b669d9de481e2ed7a58 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/acde3b42e0a24b669d9de481e2ed7a58 2023-05-05 15:55:28,771 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/ed160bbb97b0443795d0c30775736dc0 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/ed160bbb97b0443795d0c30775736dc0 2023-05-05 15:55:28,773 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/f1739bfafb3645d5a7cc0983a90af737 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf1/f1739bfafb3645d5a7cc0983a90af737 2023-05-05 15:55:28,775 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/08fd52360b9f4a17b4780fd97344db08 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/08fd52360b9f4a17b4780fd97344db08 2023-05-05 15:55:28,776 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/1eddb2bbca224552bf6f36c75678a48f to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/1eddb2bbca224552bf6f36c75678a48f 2023-05-05 15:55:28,777 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/8360124304a6402bac5e11aca7902e99 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/8360124304a6402bac5e11aca7902e99 2023-05-05 15:55:28,778 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/8c9ed392fcd34fd59ff49859c3e4924f to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/8c9ed392fcd34fd59ff49859c3e4924f 2023-05-05 15:55:28,779 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/9cf9dcbd6a8b4f7dafa8a62eb3b45f0b to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/9cf9dcbd6a8b4f7dafa8a62eb3b45f0b 2023-05-05 15:55:28,780 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/a60e6b5fcbe54c429c7d4ea740445f73 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/a60e6b5fcbe54c429c7d4ea740445f73 2023-05-05 15:55:28,781 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/ce273ec523884f0bb9bbbaae1101bf9a to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/cf2/ce273ec523884f0bb9bbbaae1101bf9a 2023-05-05 15:55:28,784 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/recovered.edits/85.seqid to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c/recovered.edits/85.seqid 2023-05-05 15:55:28,784 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegionEmptyDaughter/94b959e0a954a0b5ed4ceaeaa080b44c 2023-05-05 15:55:28,784 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-05-05 15:55:28,787 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-05 15:55:28,789 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-05-05 15:55:28,791 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-05-05 15:55:28,792 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-05 15:55:28,792 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-05-05 15:55:28,792 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302128792"}]},"ts":"9223372036854775807"} 2023-05-05 15:55:28,792 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302128792"}]},"ts":"9223372036854775807"} 2023-05-05 15:55:28,792 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1683302127569.818a1268dceef7f73d6ac0ae10bbd67e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302128792"}]},"ts":"9223372036854775807"} 2023-05-05 15:55:28,794 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-05 15:55:28,794 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 94b959e0a954a0b5ed4ceaeaa080b44c, NAME => 'testSplitTableRegionEmptyDaughter,,1683302126026.94b959e0a954a0b5ed4ceaeaa080b44c.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 319ab08408008171bab242968a65d339, NAME => 'testSplitTableRegionEmptyDaughter,,1683302127569.319ab08408008171bab242968a65d339.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 818a1268dceef7f73d6ac0ae10bbd67e, NAME => 'testSplitTableRegionEmptyDaughter,71,1683302127569.818a1268dceef7f73d6ac0ae10bbd67e.', STARTKEY => '71', ENDKEY => ''}] 2023-05-05 15:55:28,794 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-05-05 15:55:28,794 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683302128794"}]},"ts":"9223372036854775807"} 2023-05-05 15:55:28,795 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-05-05 15:55:28,798 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-05 15:55:28,799 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 49 msec 2023-05-05 15:55:28,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-05-05 15:55:28,852 INFO [Listener at localhost/35117] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-05-05 15:55:28,868 INFO [Listener at localhost/35117] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=377 (was 377), OpenFileDescriptor=859 (was 859), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=261 (was 275), ProcessCount=179 (was 179), AvailableMemoryMB=5106 (was 5110) 2023-05-05 15:55:28,883 INFO [Listener at localhost/35117] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=377, OpenFileDescriptor=859, MaxFileDescriptor=60000, SystemLoadAverage=261, ProcessCount=179, AvailableMemoryMB=5106 2023-05-05 15:55:28,883 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-05 15:55:28,884 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-05 15:55:28,885 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-05 15:55:28,886 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-05-05 15:55:28,887 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-05-05 15:55:28,888 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-05 15:55:28,889 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:28,890 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7 empty. 2023-05-05 15:55:28,890 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:28,890 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-05-05 15:55:28,903 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-05-05 15:55:28,904 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7675): creating {ENCODED => 53274b7324db593bcdb14d5b85f1c6b7, NAME => 'testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp 2023-05-05 15:55:28,911 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:55:28,911 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1604): Closing 53274b7324db593bcdb14d5b85f1c6b7, disabling compactions & flushes 2023-05-05 15:55:28,911 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. 2023-05-05 15:55:28,911 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. 2023-05-05 15:55:28,911 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. after waiting 0 ms 2023-05-05 15:55:28,911 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. 2023-05-05 15:55:28,912 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. 2023-05-05 15:55:28,912 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1558): Region close journal for 53274b7324db593bcdb14d5b85f1c6b7: 2023-05-05 15:55:28,913 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-05-05 15:55:28,914 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683302128914"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302128914"}]},"ts":"1683302128914"} 2023-05-05 15:55:28,915 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-05 15:55:28,916 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-05 15:55:28,916 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302128916"}]},"ts":"1683302128916"} 2023-05-05 15:55:28,917 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-05-05 15:55:28,919 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-05 15:55:28,920 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-05 15:55:28,920 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-05 15:55:28,920 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-05 15:55:28,920 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-05 15:55:28,921 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=53274b7324db593bcdb14d5b85f1c6b7, ASSIGN}] 2023-05-05 15:55:28,922 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=53274b7324db593bcdb14d5b85f1c6b7, ASSIGN 2023-05-05 15:55:28,922 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=53274b7324db593bcdb14d5b85f1c6b7, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40991,1683302023016; forceNewPlan=false, retain=false 2023-05-05 15:55:29,072 INFO [jenkins-hbase4:44213] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-05 15:55:29,073 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=53274b7324db593bcdb14d5b85f1c6b7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,074 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683302129073"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302129073"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302129073"}]},"ts":"1683302129073"} 2023-05-05 15:55:29,075 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure 53274b7324db593bcdb14d5b85f1c6b7, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:55:29,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. 2023-05-05 15:55:29,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 53274b7324db593bcdb14d5b85f1c6b7, NAME => 'testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7.', STARTKEY => '', ENDKEY => ''} 2023-05-05 15:55:29,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:55:29,230 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:29,230 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:55:29,230 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:29,230 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:29,231 INFO [StoreOpener-53274b7324db593bcdb14d5b85f1c6b7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:29,232 DEBUG [StoreOpener-53274b7324db593bcdb14d5b85f1c6b7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1 2023-05-05 15:55:29,232 DEBUG [StoreOpener-53274b7324db593bcdb14d5b85f1c6b7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1 2023-05-05 15:55:29,232 INFO [StoreOpener-53274b7324db593bcdb14d5b85f1c6b7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 53274b7324db593bcdb14d5b85f1c6b7 columnFamilyName cf1 2023-05-05 15:55:29,233 INFO [StoreOpener-53274b7324db593bcdb14d5b85f1c6b7-1] regionserver.HStore(310): Store=53274b7324db593bcdb14d5b85f1c6b7/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:55:29,233 INFO [StoreOpener-53274b7324db593bcdb14d5b85f1c6b7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:29,234 DEBUG [StoreOpener-53274b7324db593bcdb14d5b85f1c6b7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2 2023-05-05 15:55:29,234 DEBUG [StoreOpener-53274b7324db593bcdb14d5b85f1c6b7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2 2023-05-05 15:55:29,234 INFO [StoreOpener-53274b7324db593bcdb14d5b85f1c6b7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 53274b7324db593bcdb14d5b85f1c6b7 columnFamilyName cf2 2023-05-05 15:55:29,234 INFO [StoreOpener-53274b7324db593bcdb14d5b85f1c6b7-1] regionserver.HStore(310): Store=53274b7324db593bcdb14d5b85f1c6b7/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:55:29,235 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:29,235 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:29,237 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-05 15:55:29,237 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:29,239 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-05 15:55:29,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 53274b7324db593bcdb14d5b85f1c6b7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67910219, jitterRate=0.011941120028495789}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-05 15:55:29,239 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 53274b7324db593bcdb14d5b85f1c6b7: 2023-05-05 15:55:29,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7., pid=143, masterSystemTime=1683302129226 2023-05-05 15:55:29,241 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. 2023-05-05 15:55:29,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. 2023-05-05 15:55:29,242 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=53274b7324db593bcdb14d5b85f1c6b7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,242 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683302129242"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302129242"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302129242"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302129242"}]},"ts":"1683302129242"} 2023-05-05 15:55:29,244 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-05-05 15:55:29,244 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure 53274b7324db593bcdb14d5b85f1c6b7, server=jenkins-hbase4.apache.org,40991,1683302023016 in 168 msec 2023-05-05 15:55:29,245 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-05-05 15:55:29,245 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=53274b7324db593bcdb14d5b85f1c6b7, ASSIGN in 323 msec 2023-05-05 15:55:29,246 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-05 15:55:29,246 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302129246"}]},"ts":"1683302129246"} 2023-05-05 15:55:29,247 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-05-05 15:55:29,249 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-05-05 15:55:29,250 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 365 msec 2023-05-05 15:55:29,393 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:55:29,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-05-05 15:55:29,394 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-05-05 15:55:29,394 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:29,394 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-05-05 15:55:29,394 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-05-05 15:55:29,395 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,395 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,396 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:29,396 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:29,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:29,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:29,396 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:29,396 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:29,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:29,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:29,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:29,396 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,396 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:55:29,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,397 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,397 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,397 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:29,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,397 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-05 15:55:29,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:29,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:29,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,397 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-05 15:55:29,397 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-05 15:55:29,397 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-05 15:55:29,397 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-05 15:55:29,397 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:29,398 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-05 15:55:29,398 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-05 15:55:29,398 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-05 15:55:29,398 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-05 15:55:29,398 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-05 15:55:29,398 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-05 15:55:29,398 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-05 15:55:29,398 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:29,398 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:29,398 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-05 15:55:29,398 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-05 15:55:29,398 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-05 15:55:29,398 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-05 15:55:29,398 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:55:29,398 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:55:29,398 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. 2023-05-05 15:55:29,398 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. started... 2023-05-05 15:55:29,399 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 53274b7324db593bcdb14d5b85f1c6b7 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-05 15:55:29,399 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,399 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,400 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:29,400 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,401 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,401 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:29,409 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf1/363d4c71262f41128d69f06264c2c9dc 2023-05-05 15:55:29,420 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf2/4d6df678fe1543a9b597b5b483c413b9 2023-05-05 15:55:29,424 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf1/363d4c71262f41128d69f06264c2c9dc as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/363d4c71262f41128d69f06264c2c9dc 2023-05-05 15:55:29,427 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/363d4c71262f41128d69f06264c2c9dc, entries=2, sequenceid=6, filesize=4.8 K 2023-05-05 15:55:29,428 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf2/4d6df678fe1543a9b597b5b483c413b9 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/4d6df678fe1543a9b597b5b483c413b9 2023-05-05 15:55:29,430 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/4d6df678fe1543a9b597b5b483c413b9, entries=2, sequenceid=6, filesize=4.8 K 2023-05-05 15:55:29,431 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 53274b7324db593bcdb14d5b85f1c6b7 in 32ms, sequenceid=6, compaction requested=false 2023-05-05 15:55:29,431 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-05-05 15:55:29,432 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 53274b7324db593bcdb14d5b85f1c6b7: 2023-05-05 15:55:29,432 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. 2023-05-05 15:55:29,432 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:55:29,432 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:55:29,432 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,432 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-05 15:55:29,432 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-05 15:55:29,435 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,435 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:29,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:29,435 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,435 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:29,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:29,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:29,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:29,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:29,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:29,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:29,437 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-05-05 15:55:29,437 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3c38646b[Count = 0] remaining members to acquire global barrier 2023-05-05 15:55:29,437 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-05-05 15:55:29,437 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,438 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,438 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,438 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,438 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,438 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:55:29,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,438 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-05 15:55:29,438 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-05 15:55:29,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,438 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:55:29,438 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-05 15:55:29,438 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-05 15:55:29,438 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:55:29,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,438 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-05 15:55:29,438 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-05 15:55:29,438 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:55:29,439 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-05 15:55:29,440 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:29,440 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-05 15:55:29,443 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,443 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-05 15:55:29,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:29,443 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-05 15:55:29,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:29,443 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:29,444 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:29,444 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-05 15:55:29,444 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-05 15:55:29,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:29,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:29,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:29,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:29,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:29,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:29,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:29,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:29,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:29,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:55:29,446 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-05-05 15:55:29,446 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-05-05 15:55:29,446 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:55:29,446 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:55:29,446 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-05-05 15:55:29,446 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:55:29,448 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,448 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,448 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:29,448 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,448 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:29,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:29,448 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:29,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,448 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:29,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:29,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:29,448 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:29,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:29,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:29,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:29,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:29,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:29,449 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:29,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:29,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,449 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:29,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:29,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:29,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:29,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,451 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:29,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:29,451 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:29,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:29,451 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:29,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:29,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,454 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,454 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:29,454 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:29,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:29,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:29,454 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:29,454 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:29,454 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:29,454 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,454 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:29,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:29,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:29,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:29,454 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-05-05 15:55:29,454 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:55:29,454 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:29,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:29,454 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:29,454 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:29,454 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:29,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:29,454 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-05-05 15:55:29,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:29,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:29,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:29,454 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:29,455 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:55:29,455 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,455 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:29,455 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:29,455 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,455 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,555 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:55:29,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:55:29,566 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:55:29,567 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-05-05 15:55:29,567 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-05-05 15:55:29,567 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-05-05 15:55:29,567 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:29,567 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-05-05 15:55:29,567 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-05-05 15:55:29,568 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,568 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,569 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:29,569 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:29,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:29,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:29,569 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,569 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:29,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:29,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:29,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:29,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:29,569 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,569 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:55:29,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,570 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,570 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,570 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:29,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,570 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-05 15:55:29,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:29,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:29,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,570 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-05 15:55:29,570 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-05 15:55:29,570 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-05 15:55:29,570 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-05 15:55:29,571 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-05 15:55:29,571 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:29,571 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-05 15:55:29,571 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-05 15:55:29,571 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-05 15:55:29,571 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-05 15:55:29,571 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-05 15:55:29,571 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-05 15:55:29,571 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-05 15:55:29,571 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:29,571 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-05 15:55:29,571 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:29,571 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-05 15:55:29,571 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-05 15:55:29,571 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-05 15:55:29,571 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-05 15:55:29,571 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-05 15:55:29,571 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:55:29,571 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:55:29,571 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. 2023-05-05 15:55:29,572 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. started... 2023-05-05 15:55:29,572 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 53274b7324db593bcdb14d5b85f1c6b7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:55:29,574 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,575 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,575 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,575 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:29,575 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,575 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:29,585 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf1/86e9b67593734675a3e910cdbc0caf13 2023-05-05 15:55:29,600 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf2/587768fab2c34b5b891d6cdd8e5d2497 2023-05-05 15:55:29,604 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf1/86e9b67593734675a3e910cdbc0caf13 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/86e9b67593734675a3e910cdbc0caf13 2023-05-05 15:55:29,607 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/86e9b67593734675a3e910cdbc0caf13, entries=10, sequenceid=19, filesize=5.0 K 2023-05-05 15:55:29,608 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf2/587768fab2c34b5b891d6cdd8e5d2497 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/587768fab2c34b5b891d6cdd8e5d2497 2023-05-05 15:55:29,611 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/587768fab2c34b5b891d6cdd8e5d2497, entries=10, sequenceid=19, filesize=5.0 K 2023-05-05 15:55:29,612 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 53274b7324db593bcdb14d5b85f1c6b7 in 40ms, sequenceid=19, compaction requested=false 2023-05-05 15:55:29,612 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 53274b7324db593bcdb14d5b85f1c6b7: 2023-05-05 15:55:29,612 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. 2023-05-05 15:55:29,612 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:55:29,612 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:55:29,612 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,612 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-05 15:55:29,612 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-05 15:55:29,614 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,614 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:29,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:29,614 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:29,614 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:29,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:29,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:29,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:29,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:29,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:29,616 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-05-05 15:55:29,616 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5fef2f8e[Count = 0] remaining members to acquire global barrier 2023-05-05 15:55:29,616 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-05-05 15:55:29,616 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,617 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,617 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,617 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,617 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,617 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:55:29,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,617 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-05 15:55:29,617 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-05 15:55:29,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,617 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-05 15:55:29,617 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-05 15:55:29,617 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:55:29,617 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:55:29,617 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-05 15:55:29,618 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-05 15:55:29,618 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:55:29,619 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-05 15:55:29,619 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:29,619 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-05 15:55:29,620 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-05 15:55:29,620 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,620 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-05 15:55:29,620 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:29,620 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:29,620 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-05 15:55:29,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:29,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:29,620 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-05 15:55:29,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:29,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:29,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:29,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:29,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:29,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:29,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:29,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:29,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:29,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:55:29,623 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-05-05 15:55:29,623 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-05-05 15:55:29,623 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:55:29,623 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:55:29,623 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-05-05 15:55:29,623 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:55:29,624 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,624 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,625 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,625 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,624 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:29,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:29,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:29,625 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:29,625 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:29,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:29,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:29,625 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:29,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:29,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:29,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:29,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,626 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:29,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:29,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:29,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:29,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,626 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:29,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:29,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:29,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:29,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,627 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:29,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:29,627 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:29,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:29,628 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:29,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:29,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,630 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:29,630 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:29,630 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:29,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:29,630 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:29,630 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:29,630 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,630 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-05-05 15:55:29,630 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:55:29,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:29,630 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:29,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:29,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:29,630 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:29,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:29,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:29,630 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:29,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:29,631 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-05-05 15:55:29,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:29,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:29,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:29,631 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:29,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:29,631 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:55:29,631 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:29,631 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,631 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,631 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,631 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:29,632 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:29,632 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,632 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,731 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:55:29,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:55:29,743 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:55:29,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-05-05 15:55:29,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-05-05 15:55:29,744 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-05-05 15:55:29,744 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:29,744 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-05-05 15:55:29,744 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-05-05 15:55:29,745 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,745 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,747 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:29,747 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:29,747 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:29,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:29,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:29,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:29,747 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:29,747 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:29,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:29,748 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:55:29,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:29,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,748 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,748 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,748 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:29,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:29,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:29,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,748 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-05 15:55:29,748 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-05 15:55:29,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:29,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:29,749 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-05 15:55:29,749 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-05 15:55:29,749 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-05 15:55:29,749 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-05 15:55:29,749 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-05 15:55:29,749 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-05 15:55:29,749 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-05 15:55:29,749 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:29,749 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-05 15:55:29,749 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-05 15:55:29,749 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-05 15:55:29,749 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:29,749 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-05 15:55:29,749 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-05 15:55:29,749 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-05 15:55:29,749 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-05 15:55:29,749 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:29,749 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:55:29,749 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. 2023-05-05 15:55:29,749 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:55:29,749 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-05 15:55:29,749 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-05 15:55:29,750 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-05 15:55:29,749 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. started... 2023-05-05 15:55:29,750 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 53274b7324db593bcdb14d5b85f1c6b7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:55:29,750 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,751 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,751 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:29,755 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,755 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:29,755 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:30,160 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf1/21ca3c427d50442da6f1361f29796bb7 2023-05-05 15:55:30,171 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf2/b34e6f3dba14493cac9ced4f4566b2b7 2023-05-05 15:55:30,175 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf1/21ca3c427d50442da6f1361f29796bb7 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/21ca3c427d50442da6f1361f29796bb7 2023-05-05 15:55:30,178 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/21ca3c427d50442da6f1361f29796bb7, entries=10, sequenceid=32, filesize=5.0 K 2023-05-05 15:55:30,179 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf2/b34e6f3dba14493cac9ced4f4566b2b7 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/b34e6f3dba14493cac9ced4f4566b2b7 2023-05-05 15:55:30,182 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/b34e6f3dba14493cac9ced4f4566b2b7, entries=10, sequenceid=32, filesize=5.0 K 2023-05-05 15:55:30,183 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 53274b7324db593bcdb14d5b85f1c6b7 in 433ms, sequenceid=32, compaction requested=true 2023-05-05 15:55:30,183 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 53274b7324db593bcdb14d5b85f1c6b7: 2023-05-05 15:55:30,183 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. 2023-05-05 15:55:30,183 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:55:30,183 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:55:30,183 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,183 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-05 15:55:30,183 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-05 15:55:30,185 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,185 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:30,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:30,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:30,185 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,185 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:30,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:30,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:30,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:30,186 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-05-05 15:55:30,186 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1536a3d2[Count = 0] remaining members to acquire global barrier 2023-05-05 15:55:30,186 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-05-05 15:55:30,186 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,187 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,187 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,187 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,188 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-05 15:55:30,188 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-05 15:55:30,188 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:55:30,188 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,188 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:55:30,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,188 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-05 15:55:30,188 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-05 15:55:30,188 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-05 15:55:30,188 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-05 15:55:30,188 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:55:30,188 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:55:30,190 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-05 15:55:30,190 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:30,190 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-05 15:55:30,191 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-05 15:55:30,191 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,191 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-05 15:55:30,191 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:30,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,191 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-05 15:55:30,191 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:30,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:30,191 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-05 15:55:30,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:30,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:30,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:30,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:30,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:30,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:30,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:55:30,194 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-05-05 15:55:30,194 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-05-05 15:55:30,194 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:55:30,194 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:55:30,194 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-05-05 15:55:30,194 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:55:30,195 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,195 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,195 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:30,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:30,195 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:30,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,195 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:30,195 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,196 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:30,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:30,196 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:30,196 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:30,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:30,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:30,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:30,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:30,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:30,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,197 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:30,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:30,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:30,198 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:30,199 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,199 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,201 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,201 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,201 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:30,201 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,201 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:30,201 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:30,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:30,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:30,201 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:30,201 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-05-05 15:55:30,201 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:55:30,201 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:30,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:30,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:30,201 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:30,201 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:30,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:30,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:30,202 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:30,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:30,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:30,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:30,201 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,202 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-05-05 15:55:30,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:30,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:30,202 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,202 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:55:30,202 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,202 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,202 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,202 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,202 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,302 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:55:30,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:55:30,313 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:55:30,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-05-05 15:55:30,315 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-05-05 15:55:30,315 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-05-05 15:55:30,315 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:30,315 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-05-05 15:55:30,315 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-05-05 15:55:30,315 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,315 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,318 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:30,318 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:30,318 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:30,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:30,318 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:30,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:30,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:30,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:30,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:30,318 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,318 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:55:30,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,318 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,319 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,319 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:30,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,319 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-05 15:55:30,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:30,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,319 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-05 15:55:30,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:30,319 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-05 15:55:30,319 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-05 15:55:30,319 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-05 15:55:30,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,320 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-05 15:55:30,320 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-05 15:55:30,319 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-05 15:55:30,320 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:30,320 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-05 15:55:30,320 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:30,320 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-05 15:55:30,320 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-05 15:55:30,320 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-05 15:55:30,320 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-05 15:55:30,320 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-05 15:55:30,320 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-05 15:55:30,320 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-05 15:55:30,320 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-05 15:55:30,320 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-05 15:55:30,320 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:30,320 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-05 15:55:30,320 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:55:30,320 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:55:30,320 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. 2023-05-05 15:55:30,320 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. started... 2023-05-05 15:55:30,321 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 53274b7324db593bcdb14d5b85f1c6b7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:55:30,321 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,321 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,321 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,322 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:30,322 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,322 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:30,331 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf1/0a830604edc34189ae5830c6f54495a4 2023-05-05 15:55:30,346 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf2/3acc3bde4ad64554b6f92e4994b2ea0d 2023-05-05 15:55:30,349 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf1/0a830604edc34189ae5830c6f54495a4 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/0a830604edc34189ae5830c6f54495a4 2023-05-05 15:55:30,352 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/0a830604edc34189ae5830c6f54495a4, entries=10, sequenceid=45, filesize=5.0 K 2023-05-05 15:55:30,353 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf2/3acc3bde4ad64554b6f92e4994b2ea0d as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/3acc3bde4ad64554b6f92e4994b2ea0d 2023-05-05 15:55:30,357 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/3acc3bde4ad64554b6f92e4994b2ea0d, entries=10, sequenceid=45, filesize=5.0 K 2023-05-05 15:55:30,357 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 53274b7324db593bcdb14d5b85f1c6b7 in 36ms, sequenceid=45, compaction requested=true 2023-05-05 15:55:30,357 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 53274b7324db593bcdb14d5b85f1c6b7: 2023-05-05 15:55:30,357 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. 2023-05-05 15:55:30,357 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:55:30,357 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:55:30,358 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,358 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-05 15:55:30,358 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-05 15:55:30,359 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:30,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:30,359 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:30,359 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,359 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:30,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:30,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:30,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:30,361 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-05-05 15:55:30,361 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b7bad28[Count = 0] remaining members to acquire global barrier 2023-05-05 15:55:30,361 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-05-05 15:55:30,361 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,362 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,362 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,362 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,362 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-05 15:55:30,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,362 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,362 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-05 15:55:30,362 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-05 15:55:30,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,363 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:55:30,362 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:55:30,362 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-05 15:55:30,363 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-05 15:55:30,363 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-05 15:55:30,363 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:55:30,363 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:55:30,366 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-05 15:55:30,366 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:30,366 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-05 15:55:30,367 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-05 15:55:30,367 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:30,367 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-05 15:55:30,367 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-05 15:55:30,367 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:30,367 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-05 15:55:30,368 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:30,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:30,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:30,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:30,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:30,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:30,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:30,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:55:30,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-05-05 15:55:30,371 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-05-05 15:55:30,372 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:55:30,372 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:55:30,372 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-05-05 15:55:30,372 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:55:30,373 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,373 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,373 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,373 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,373 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:30,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,373 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:30,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,374 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:30,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:30,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:30,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:30,373 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:30,374 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:30,374 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:30,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:30,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:30,374 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:30,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,375 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:30,375 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:30,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:30,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:30,376 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:30,377 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,377 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,383 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,383 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,383 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:30,383 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:30,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:30,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:30,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:30,383 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-05-05 15:55:30,384 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:55:30,383 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:30,383 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:30,383 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:30,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:30,384 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,384 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:30,383 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:30,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:30,383 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:30,384 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,384 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-05-05 15:55:30,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:30,384 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:55:30,384 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:30,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:30,384 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,385 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,385 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,385 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,385 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:30,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:30,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:30,484 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:55:30,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:55:30,495 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:55:30,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-05-05 15:55:30,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-05-05 15:55:30,496 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-05-05 15:55:30,496 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:30,496 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-05-05 15:55:30,496 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-05-05 15:55:30,497 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,497 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,498 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:30,498 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:30,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:30,498 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,498 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:30,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:30,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:30,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:30,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:30,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:30,498 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,498 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:55:30,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,499 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,499 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,499 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:30,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,499 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-05 15:55:30,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:30,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:30,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,499 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-05 15:55:30,499 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-05 15:55:30,499 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-05 15:55:30,500 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-05 15:55:30,500 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-05 15:55:30,500 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-05 15:55:30,500 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-05 15:55:30,500 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:30,500 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-05 15:55:30,500 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-05 15:55:30,500 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-05 15:55:30,500 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-05 15:55:30,500 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:30,500 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:30,500 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-05 15:55:30,500 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-05 15:55:30,500 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-05 15:55:30,500 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-05 15:55:30,500 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-05 15:55:30,500 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-05 15:55:30,500 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:55:30,500 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. 2023-05-05 15:55:30,500 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-05 15:55:30,500 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. started... 2023-05-05 15:55:30,500 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:55:30,501 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 53274b7324db593bcdb14d5b85f1c6b7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:55:30,502 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,502 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,502 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,502 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:30,502 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,502 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:30,509 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf1/0af86f1ef5df40078761b720c0239432 2023-05-05 15:55:30,522 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf2/8a63f8737720490395ee63cac7fd38b1 2023-05-05 15:55:30,525 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf1/0af86f1ef5df40078761b720c0239432 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/0af86f1ef5df40078761b720c0239432 2023-05-05 15:55:30,529 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/0af86f1ef5df40078761b720c0239432, entries=10, sequenceid=58, filesize=5.0 K 2023-05-05 15:55:30,529 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf2/8a63f8737720490395ee63cac7fd38b1 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/8a63f8737720490395ee63cac7fd38b1 2023-05-05 15:55:30,532 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/8a63f8737720490395ee63cac7fd38b1, entries=10, sequenceid=58, filesize=5.0 K 2023-05-05 15:55:30,533 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 53274b7324db593bcdb14d5b85f1c6b7 in 32ms, sequenceid=58, compaction requested=true 2023-05-05 15:55:30,533 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 53274b7324db593bcdb14d5b85f1c6b7: 2023-05-05 15:55:30,533 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. 2023-05-05 15:55:30,533 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:55:30,533 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:55:30,533 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,533 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-05 15:55:30,533 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-05 15:55:30,535 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,535 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:30,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:30,536 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:30,536 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:30,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:30,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:30,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:30,537 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-05-05 15:55:30,537 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@649f8910[Count = 0] remaining members to acquire global barrier 2023-05-05 15:55:30,537 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-05-05 15:55:30,538 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,539 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,539 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,539 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,539 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,539 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:55:30,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,539 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-05 15:55:30,539 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-05 15:55:30,539 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-05 15:55:30,539 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:55:30,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,539 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-05 15:55:30,539 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-05 15:55:30,539 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-05 15:55:30,539 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:55:30,539 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:55:30,540 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-05 15:55:30,540 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:30,540 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-05 15:55:30,541 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-05 15:55:30,541 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,541 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-05 15:55:30,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:30,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:30,541 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:30,541 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:30,541 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-05 15:55:30,541 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-05 15:55:30,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:30,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:30,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:30,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:30,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:30,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:55:30,544 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-05-05 15:55:30,544 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-05-05 15:55:30,544 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:55:30,544 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:55:30,544 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-05-05 15:55:30,544 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:55:30,547 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,547 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,547 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,547 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:30,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:30,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,547 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:30,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,547 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:30,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,547 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:30,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:30,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:30,547 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:30,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:30,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:30,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:30,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:30,548 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:30,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:30,554 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:30,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,555 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:30,556 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:30,556 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,559 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,559 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,559 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:30,559 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:30,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:30,559 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,559 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:30,559 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:30,559 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,559 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:30,559 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:30,559 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:30,559 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-05-05 15:55:30,559 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:55:30,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:30,559 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:30,559 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:30,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:30,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:30,559 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:30,560 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:30,560 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-05-05 15:55:30,560 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:55:30,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:30,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:30,560 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,560 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,560 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:30,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:30,660 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:55:30,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:55:30,673 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-05 15:55:30,674 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-05-05 15:55:30,674 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-05-05 15:55:30,674 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-05-05 15:55:30,674 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:30,674 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-05-05 15:55:30,674 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-05-05 15:55:30,675 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,675 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,676 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:30,676 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,676 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:30,676 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:30,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:30,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:30,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:30,676 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,676 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-05 15:55:30,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:30,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:30,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:30,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,677 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,677 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,677 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:30,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:30,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,678 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-05 15:55:30,678 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-05 15:55:30,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-05 15:55:30,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,678 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-05 15:55:30,678 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-05 15:55:30,678 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-05 15:55:30,678 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-05 15:55:30,678 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:30,678 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-05 15:55:30,678 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-05 15:55:30,678 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43023,1683302022922' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-05 15:55:30,678 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-05 15:55:30,679 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-05 15:55:30,679 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-05 15:55:30,679 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-05 15:55:30,679 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:30,679 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-05 15:55:30,679 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-05 15:55:30,679 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-05 15:55:30,679 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-05 15:55:30,679 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-05 15:55:30,679 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-05 15:55:30,679 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-05 15:55:30,679 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-05 15:55:30,679 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. 2023-05-05 15:55:30,679 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42799,1683302022977' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-05 15:55:30,679 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. started... 2023-05-05 15:55:30,679 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 53274b7324db593bcdb14d5b85f1c6b7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-05 15:55:30,680 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,682 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,682 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,682 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:30,682 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,682 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:30,688 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf1/65bb1d6cc3f2438fa61cda724ebd2b75 2023-05-05 15:55:30,698 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf2/fd12d1c07c8d41ff83a744ea02f34eb5 2023-05-05 15:55:30,701 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf1/65bb1d6cc3f2438fa61cda724ebd2b75 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/65bb1d6cc3f2438fa61cda724ebd2b75 2023-05-05 15:55:30,704 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/65bb1d6cc3f2438fa61cda724ebd2b75, entries=10, sequenceid=71, filesize=5.0 K 2023-05-05 15:55:30,705 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf2/fd12d1c07c8d41ff83a744ea02f34eb5 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/fd12d1c07c8d41ff83a744ea02f34eb5 2023-05-05 15:55:30,708 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/fd12d1c07c8d41ff83a744ea02f34eb5, entries=10, sequenceid=71, filesize=5.0 K 2023-05-05 15:55:30,708 INFO [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 53274b7324db593bcdb14d5b85f1c6b7 in 29ms, sequenceid=71, compaction requested=true 2023-05-05 15:55:30,708 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 53274b7324db593bcdb14d5b85f1c6b7: 2023-05-05 15:55:30,708 DEBUG [rs(jenkins-hbase4.apache.org,40991,1683302023016)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. 2023-05-05 15:55:30,708 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-05 15:55:30,708 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-05 15:55:30,709 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,709 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-05 15:55:30,709 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-05 15:55:30,710 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,710 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:30,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:30,710 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,710 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-05 15:55:30,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:30,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:30,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:30,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:30,712 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40991,1683302023016' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-05-05 15:55:30,712 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@42aa5952[Count = 0] remaining members to acquire global barrier 2023-05-05 15:55:30,712 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-05-05 15:55:30,712 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,713 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,713 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,714 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,714 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-05 15:55:30,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,714 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-05 15:55:30,714 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-05 15:55:30,714 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-05 15:55:30,714 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-05 15:55:30,714 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40991,1683302023016' in zk 2023-05-05 15:55:30,714 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42799,1683302022977' in zk 2023-05-05 15:55:30,718 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-05 15:55:30,718 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-05 15:55:30,718 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,718 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:30,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,718 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:30,718 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,719 DEBUG [member: 'jenkins-hbase4.apache.org,42799,1683302022977' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-05 15:55:30,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:30,719 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-05 15:55:30,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:30,719 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-05 15:55:30,719 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43023,1683302022922' in zk 2023-05-05 15:55:30,719 DEBUG [member: 'jenkins-hbase4.apache.org,40991,1683302023016' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-05 15:55:30,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:30,720 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-05 15:55:30,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:30,720 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:30,720 DEBUG [member: 'jenkins-hbase4.apache.org,43023,1683302022922' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-05 15:55:30,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:30,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:30,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:30,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,40991,1683302023016': 2023-05-05 15:55:30,722 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40991,1683302023016' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-05-05 15:55:30,722 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-05-05 15:55:30,722 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-05 15:55:30,722 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-05 15:55:30,722 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-05-05 15:55:30,722 INFO [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-05 15:55:30,724 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,724 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,724 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,724 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,724 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:30,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,724 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:30,724 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:30,724 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-05 15:55:30,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-05 15:55:30,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:30,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:30,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:30,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:30,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:30,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:30,724 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-05 15:55:30,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,725 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:30,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-05 15:55:30,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:30,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,726 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-05 15:55:30,726 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-05 15:55:30,727 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,730 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,730 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:30,730 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,730 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:30,730 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,730 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:30,730 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:30,730 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:30,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-05-05 15:55:30,731 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-05 15:55:30,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:30,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:30,730 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-05 15:55:30,730 DEBUG [(jenkins-hbase4.apache.org,44213,1683302021756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-05 15:55:30,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:30,730 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-05 15:55:30,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-05 15:55:30,731 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-05-05 15:55:30,731 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-05 15:55:30,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:30,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:30,731 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-05 15:55:30,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:30,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:30,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-05 15:55:30,731 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:30,731 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,731 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,731 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:30,731 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-05 15:55:30,732 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-05 15:55:30,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-05 15:55:30,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-05 15:55:30,831 DEBUG [Listener at localhost/35117] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-05 15:55:30,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-05 15:55:30,840 DEBUG [Listener at localhost/35117] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,840 DEBUG [Listener at localhost/35117] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=53274b7324db593bcdb14d5b85f1c6b7, daughterA=e84c6af08e7e6bceea057422cc6e6cbe, daughterB=9b5dcde66b10fccd0c9178f6827394e9 2023-05-05 15:55:30,842 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=53274b7324db593bcdb14d5b85f1c6b7, daughterA=e84c6af08e7e6bceea057422cc6e6cbe, daughterB=9b5dcde66b10fccd0c9178f6827394e9 2023-05-05 15:55:30,842 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=53274b7324db593bcdb14d5b85f1c6b7, daughterA=e84c6af08e7e6bceea057422cc6e6cbe, daughterB=9b5dcde66b10fccd0c9178f6827394e9 2023-05-05 15:55:30,842 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=53274b7324db593bcdb14d5b85f1c6b7, daughterA=e84c6af08e7e6bceea057422cc6e6cbe, daughterB=9b5dcde66b10fccd0c9178f6827394e9 2023-05-05 15:55:30,843 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=53274b7324db593bcdb14d5b85f1c6b7, UNASSIGN}] 2023-05-05 15:55:30,844 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=53274b7324db593bcdb14d5b85f1c6b7, UNASSIGN 2023-05-05 15:55:30,844 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=53274b7324db593bcdb14d5b85f1c6b7, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:30,845 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683302130844"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302130844"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302130844"}]},"ts":"1683302130844"} 2023-05-05 15:55:30,845 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure 53274b7324db593bcdb14d5b85f1c6b7, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:55:30,997 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:30,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 53274b7324db593bcdb14d5b85f1c6b7, disabling compactions & flushes 2023-05-05 15:55:30,997 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. 2023-05-05 15:55:30,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. 2023-05-05 15:55:30,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. after waiting 0 ms 2023-05-05 15:55:30,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. 2023-05-05 15:55:30,997 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 53274b7324db593bcdb14d5b85f1c6b7 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-05 15:55:31,006 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf1/90b8eff45a534b20af611eff0bf47cff 2023-05-05 15:55:31,016 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf2/78278f2c5a9b417da69bad0e2ad993a5 2023-05-05 15:55:31,020 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf1/90b8eff45a534b20af611eff0bf47cff as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/90b8eff45a534b20af611eff0bf47cff 2023-05-05 15:55:31,023 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/90b8eff45a534b20af611eff0bf47cff, entries=8, sequenceid=82, filesize=5.0 K 2023-05-05 15:55:31,023 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/.tmp/cf2/78278f2c5a9b417da69bad0e2ad993a5 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/78278f2c5a9b417da69bad0e2ad993a5 2023-05-05 15:55:31,026 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/78278f2c5a9b417da69bad0e2ad993a5, entries=8, sequenceid=82, filesize=5.0 K 2023-05-05 15:55:31,027 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 53274b7324db593bcdb14d5b85f1c6b7 in 30ms, sequenceid=82, compaction requested=true 2023-05-05 15:55:31,032 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-05 15:55:31,032 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:55:31,032 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. 2023-05-05 15:55:31,032 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 53274b7324db593bcdb14d5b85f1c6b7: 2023-05-05 15:55:31,033 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,034 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=53274b7324db593bcdb14d5b85f1c6b7, regionState=CLOSED 2023-05-05 15:55:31,034 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683302131034"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302131034"}]},"ts":"1683302131034"} 2023-05-05 15:55:31,036 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-05-05 15:55:31,036 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure 53274b7324db593bcdb14d5b85f1c6b7, server=jenkins-hbase4.apache.org,40991,1683302023016 in 190 msec 2023-05-05 15:55:31,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-05-05 15:55:31,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=53274b7324db593bcdb14d5b85f1c6b7, UNASSIGN in 193 msec 2023-05-05 15:55:31,045 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=53274b7324db593bcdb14d5b85f1c6b7, threads=14 2023-05-05 15:55:31,045 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/3acc3bde4ad64554b6f92e4994b2ea0d for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,046 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/4d6df678fe1543a9b597b5b483c413b9 for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,046 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/587768fab2c34b5b891d6cdd8e5d2497 for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,046 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/78278f2c5a9b417da69bad0e2ad993a5 for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,046 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/8a63f8737720490395ee63cac7fd38b1 for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,046 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/b34e6f3dba14493cac9ced4f4566b2b7 for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,046 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/fd12d1c07c8d41ff83a744ea02f34eb5 for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,046 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/0a830604edc34189ae5830c6f54495a4 for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,046 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/0af86f1ef5df40078761b720c0239432 for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,046 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/21ca3c427d50442da6f1361f29796bb7 for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,047 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/363d4c71262f41128d69f06264c2c9dc for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,047 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/65bb1d6cc3f2438fa61cda724ebd2b75 for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,047 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/86e9b67593734675a3e910cdbc0caf13 for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,047 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/90b8eff45a534b20af611eff0bf47cff for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,176 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/65bb1d6cc3f2438fa61cda724ebd2b75 for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,184 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/3acc3bde4ad64554b6f92e4994b2ea0d for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,199 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/b34e6f3dba14493cac9ced4f4566b2b7 for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,225 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/4d6df678fe1543a9b597b5b483c413b9 for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,228 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/90b8eff45a534b20af611eff0bf47cff for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,229 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/78278f2c5a9b417da69bad0e2ad993a5 for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,230 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/8a63f8737720490395ee63cac7fd38b1 for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,239 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/21ca3c427d50442da6f1361f29796bb7 for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,245 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/fd12d1c07c8d41ff83a744ea02f34eb5 for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,245 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/587768fab2c34b5b891d6cdd8e5d2497 for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,648 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/363d4c71262f41128d69f06264c2c9dc for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,648 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/86e9b67593734675a3e910cdbc0caf13 for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,649 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/0af86f1ef5df40078761b720c0239432 for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,649 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/0a830604edc34189ae5830c6f54495a4 for region: 53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:31,649 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region 53274b7324db593bcdb14d5b85f1c6b7 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-05 15:55:31,689 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-05 15:55:31,691 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-05 15:55:31,692 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683302131692"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1683302131692"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1683302131692"}]},"ts":"1683302131692"} 2023-05-05 15:55:31,692 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683302131692"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302131692"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302131692"}]},"ts":"1683302131692"} 2023-05-05 15:55:31,692 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683302131692"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302131692"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302131692"}]},"ts":"1683302131692"} 2023-05-05 15:55:31,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=e84c6af08e7e6bceea057422cc6e6cbe, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=9b5dcde66b10fccd0c9178f6827394e9, ASSIGN}] 2023-05-05 15:55:31,695 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=9b5dcde66b10fccd0c9178f6827394e9, ASSIGN 2023-05-05 15:55:31,696 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=9b5dcde66b10fccd0c9178f6827394e9, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40991,1683302023016; forceNewPlan=false, retain=false 2023-05-05 15:55:31,696 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=e84c6af08e7e6bceea057422cc6e6cbe, ASSIGN 2023-05-05 15:55:31,697 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=e84c6af08e7e6bceea057422cc6e6cbe, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40991,1683302023016; forceNewPlan=false, retain=false 2023-05-05 15:55:31,846 INFO [jenkins-hbase4:44213] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-05 15:55:31,848 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=e84c6af08e7e6bceea057422cc6e6cbe, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:31,848 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683302131848"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302131848"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302131848"}]},"ts":"1683302131848"} 2023-05-05 15:55:31,849 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=147, state=RUNNABLE; OpenRegionProcedure e84c6af08e7e6bceea057422cc6e6cbe, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:55:31,851 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=9b5dcde66b10fccd0c9178f6827394e9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:31,851 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683302131851"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302131851"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302131851"}]},"ts":"1683302131851"} 2023-05-05 15:55:31,852 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=148, state=RUNNABLE; OpenRegionProcedure 9b5dcde66b10fccd0c9178f6827394e9, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:55:32,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9. 2023-05-05 15:55:32,003 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 9b5dcde66b10fccd0c9178f6827394e9, NAME => 'testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.', STARTKEY => '41', ENDKEY => ''} 2023-05-05 15:55:32,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:55:32,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 9b5dcde66b10fccd0c9178f6827394e9 2023-05-05 15:55:32,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:55:32,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 9b5dcde66b10fccd0c9178f6827394e9 2023-05-05 15:55:32,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 9b5dcde66b10fccd0c9178f6827394e9 2023-05-05 15:55:32,005 INFO [StoreOpener-9b5dcde66b10fccd0c9178f6827394e9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 9b5dcde66b10fccd0c9178f6827394e9 2023-05-05 15:55:32,006 DEBUG [StoreOpener-9b5dcde66b10fccd0c9178f6827394e9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1 2023-05-05 15:55:32,006 DEBUG [StoreOpener-9b5dcde66b10fccd0c9178f6827394e9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1 2023-05-05 15:55:32,006 INFO [StoreOpener-9b5dcde66b10fccd0c9178f6827394e9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9b5dcde66b10fccd0c9178f6827394e9 columnFamilyName cf1 2023-05-05 15:55:32,018 DEBUG [StoreOpener-9b5dcde66b10fccd0c9178f6827394e9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/0a830604edc34189ae5830c6f54495a4.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/0a830604edc34189ae5830c6f54495a4-top 2023-05-05 15:55:32,021 DEBUG [StoreOpener-9b5dcde66b10fccd0c9178f6827394e9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/0af86f1ef5df40078761b720c0239432.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/0af86f1ef5df40078761b720c0239432-top 2023-05-05 15:55:32,024 DEBUG [StoreOpener-9b5dcde66b10fccd0c9178f6827394e9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/21ca3c427d50442da6f1361f29796bb7.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/21ca3c427d50442da6f1361f29796bb7-top 2023-05-05 15:55:32,027 DEBUG [StoreOpener-9b5dcde66b10fccd0c9178f6827394e9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/363d4c71262f41128d69f06264c2c9dc.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/363d4c71262f41128d69f06264c2c9dc-top 2023-05-05 15:55:32,030 DEBUG [StoreOpener-9b5dcde66b10fccd0c9178f6827394e9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/65bb1d6cc3f2438fa61cda724ebd2b75.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/65bb1d6cc3f2438fa61cda724ebd2b75-top 2023-05-05 15:55:32,033 DEBUG [StoreOpener-9b5dcde66b10fccd0c9178f6827394e9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/86e9b67593734675a3e910cdbc0caf13.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/86e9b67593734675a3e910cdbc0caf13-top 2023-05-05 15:55:32,036 DEBUG [StoreOpener-9b5dcde66b10fccd0c9178f6827394e9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/90b8eff45a534b20af611eff0bf47cff.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/90b8eff45a534b20af611eff0bf47cff-top 2023-05-05 15:55:32,036 INFO [StoreOpener-9b5dcde66b10fccd0c9178f6827394e9-1] regionserver.HStore(310): Store=9b5dcde66b10fccd0c9178f6827394e9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:55:32,037 INFO [StoreOpener-9b5dcde66b10fccd0c9178f6827394e9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 9b5dcde66b10fccd0c9178f6827394e9 2023-05-05 15:55:32,037 DEBUG [StoreOpener-9b5dcde66b10fccd0c9178f6827394e9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2 2023-05-05 15:55:32,037 DEBUG [StoreOpener-9b5dcde66b10fccd0c9178f6827394e9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2 2023-05-05 15:55:32,038 INFO [StoreOpener-9b5dcde66b10fccd0c9178f6827394e9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9b5dcde66b10fccd0c9178f6827394e9 columnFamilyName cf2 2023-05-05 15:55:32,048 DEBUG [StoreOpener-9b5dcde66b10fccd0c9178f6827394e9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/3acc3bde4ad64554b6f92e4994b2ea0d.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/3acc3bde4ad64554b6f92e4994b2ea0d-top 2023-05-05 15:55:32,052 DEBUG [StoreOpener-9b5dcde66b10fccd0c9178f6827394e9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/4d6df678fe1543a9b597b5b483c413b9.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/4d6df678fe1543a9b597b5b483c413b9-top 2023-05-05 15:55:32,054 DEBUG [StoreOpener-9b5dcde66b10fccd0c9178f6827394e9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/587768fab2c34b5b891d6cdd8e5d2497.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/587768fab2c34b5b891d6cdd8e5d2497-top 2023-05-05 15:55:32,057 DEBUG [StoreOpener-9b5dcde66b10fccd0c9178f6827394e9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/78278f2c5a9b417da69bad0e2ad993a5.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/78278f2c5a9b417da69bad0e2ad993a5-top 2023-05-05 15:55:32,061 DEBUG [StoreOpener-9b5dcde66b10fccd0c9178f6827394e9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/8a63f8737720490395ee63cac7fd38b1.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/8a63f8737720490395ee63cac7fd38b1-top 2023-05-05 15:55:32,064 DEBUG [StoreOpener-9b5dcde66b10fccd0c9178f6827394e9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/b34e6f3dba14493cac9ced4f4566b2b7.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/b34e6f3dba14493cac9ced4f4566b2b7-top 2023-05-05 15:55:32,066 DEBUG [StoreOpener-9b5dcde66b10fccd0c9178f6827394e9-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/fd12d1c07c8d41ff83a744ea02f34eb5.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/fd12d1c07c8d41ff83a744ea02f34eb5-top 2023-05-05 15:55:32,067 INFO [StoreOpener-9b5dcde66b10fccd0c9178f6827394e9-1] regionserver.HStore(310): Store=9b5dcde66b10fccd0c9178f6827394e9/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:55:32,067 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9 2023-05-05 15:55:32,068 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9 2023-05-05 15:55:32,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-05 15:55:32,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 9b5dcde66b10fccd0c9178f6827394e9 2023-05-05 15:55:32,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 9b5dcde66b10fccd0c9178f6827394e9; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67838334, jitterRate=0.01086995005607605}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-05 15:55:32,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 9b5dcde66b10fccd0c9178f6827394e9: 2023-05-05 15:55:32,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9., pid=150, masterSystemTime=1683302132001 2023-05-05 15:55:32,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:55:32,073 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-05 15:55:32,073 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-05 15:55:32,073 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-05 15:55:32,073 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9. 2023-05-05 15:55:32,073 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1912): 9b5dcde66b10fccd0c9178f6827394e9/cf1 is initiating minor compaction (all files) 2023-05-05 15:55:32,073 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9. 2023-05-05 15:55:32,073 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 9b5dcde66b10fccd0c9178f6827394e9/cf1 in testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9. 2023-05-05 15:55:32,073 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1912): 9b5dcde66b10fccd0c9178f6827394e9/cf2 is initiating minor compaction (all files) 2023-05-05 15:55:32,073 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 9b5dcde66b10fccd0c9178f6827394e9/cf2 in testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9. 2023-05-05 15:55:32,073 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/363d4c71262f41128d69f06264c2c9dc.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/363d4c71262f41128d69f06264c2c9dc-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/86e9b67593734675a3e910cdbc0caf13.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/86e9b67593734675a3e910cdbc0caf13-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/21ca3c427d50442da6f1361f29796bb7.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/21ca3c427d50442da6f1361f29796bb7-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/0a830604edc34189ae5830c6f54495a4.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/0a830604edc34189ae5830c6f54495a4-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/0af86f1ef5df40078761b720c0239432.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/0af86f1ef5df40078761b720c0239432-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/65bb1d6cc3f2438fa61cda724ebd2b75.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/65bb1d6cc3f2438fa61cda724ebd2b75-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/90b8eff45a534b20af611eff0bf47cff.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/90b8eff45a534b20af611eff0bf47cff-top] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/.tmp, totalSize=34.9 K 2023-05-05 15:55:32,073 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/4d6df678fe1543a9b597b5b483c413b9.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/4d6df678fe1543a9b597b5b483c413b9-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/587768fab2c34b5b891d6cdd8e5d2497.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/587768fab2c34b5b891d6cdd8e5d2497-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/b34e6f3dba14493cac9ced4f4566b2b7.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/b34e6f3dba14493cac9ced4f4566b2b7-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/3acc3bde4ad64554b6f92e4994b2ea0d.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/3acc3bde4ad64554b6f92e4994b2ea0d-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/8a63f8737720490395ee63cac7fd38b1.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/8a63f8737720490395ee63cac7fd38b1-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/fd12d1c07c8d41ff83a744ea02f34eb5.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/fd12d1c07c8d41ff83a744ea02f34eb5-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/78278f2c5a9b417da69bad0e2ad993a5.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/78278f2c5a9b417da69bad0e2ad993a5-top] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/.tmp, totalSize=34.9 K 2023-05-05 15:55:32,074 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 363d4c71262f41128d69f06264c2c9dc.53274b7324db593bcdb14d5b85f1c6b7, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683302129389 2023-05-05 15:55:32,074 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9. 2023-05-05 15:55:32,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9. 2023-05-05 15:55:32,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe. 2023-05-05 15:55:32,074 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 86e9b67593734675a3e910cdbc0caf13.53274b7324db593bcdb14d5b85f1c6b7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683302129557 2023-05-05 15:55:32,074 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 4d6df678fe1543a9b597b5b483c413b9.53274b7324db593bcdb14d5b85f1c6b7, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683302129389 2023-05-05 15:55:32,074 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e84c6af08e7e6bceea057422cc6e6cbe, NAME => 'testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.', STARTKEY => '', ENDKEY => '41'} 2023-05-05 15:55:32,074 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=9b5dcde66b10fccd0c9178f6827394e9, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:32,074 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683302132074"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302132074"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302132074"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302132074"}]},"ts":"1683302132074"} 2023-05-05 15:55:32,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-05 15:55:32,074 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 21ca3c427d50442da6f1361f29796bb7.53274b7324db593bcdb14d5b85f1c6b7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683302129733 2023-05-05 15:55:32,075 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 587768fab2c34b5b891d6cdd8e5d2497.53274b7324db593bcdb14d5b85f1c6b7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683302129557 2023-05-05 15:55:32,075 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion e84c6af08e7e6bceea057422cc6e6cbe 2023-05-05 15:55:32,075 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-05 15:55:32,075 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e84c6af08e7e6bceea057422cc6e6cbe 2023-05-05 15:55:32,075 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e84c6af08e7e6bceea057422cc6e6cbe 2023-05-05 15:55:32,075 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 0a830604edc34189ae5830c6f54495a4.53274b7324db593bcdb14d5b85f1c6b7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1683302130304 2023-05-05 15:55:32,075 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting b34e6f3dba14493cac9ced4f4566b2b7.53274b7324db593bcdb14d5b85f1c6b7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683302129733 2023-05-05 15:55:32,075 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 0af86f1ef5df40078761b720c0239432.53274b7324db593bcdb14d5b85f1c6b7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1683302130486 2023-05-05 15:55:32,075 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 3acc3bde4ad64554b6f92e4994b2ea0d.53274b7324db593bcdb14d5b85f1c6b7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1683302130304 2023-05-05 15:55:32,076 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 65bb1d6cc3f2438fa61cda724ebd2b75.53274b7324db593bcdb14d5b85f1c6b7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1683302130662 2023-05-05 15:55:32,076 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 8a63f8737720490395ee63cac7fd38b1.53274b7324db593bcdb14d5b85f1c6b7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1683302130486 2023-05-05 15:55:32,076 INFO [StoreOpener-e84c6af08e7e6bceea057422cc6e6cbe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e84c6af08e7e6bceea057422cc6e6cbe 2023-05-05 15:55:32,076 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 90b8eff45a534b20af611eff0bf47cff.53274b7324db593bcdb14d5b85f1c6b7, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683302130833 2023-05-05 15:55:32,077 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting fd12d1c07c8d41ff83a744ea02f34eb5.53274b7324db593bcdb14d5b85f1c6b7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1683302130662 2023-05-05 15:55:32,077 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 78278f2c5a9b417da69bad0e2ad993a5.53274b7324db593bcdb14d5b85f1c6b7, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683302130833 2023-05-05 15:55:32,077 DEBUG [StoreOpener-e84c6af08e7e6bceea057422cc6e6cbe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1 2023-05-05 15:55:32,077 DEBUG [StoreOpener-e84c6af08e7e6bceea057422cc6e6cbe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1 2023-05-05 15:55:32,078 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=148 2023-05-05 15:55:32,078 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=148, state=SUCCESS; OpenRegionProcedure 9b5dcde66b10fccd0c9178f6827394e9, server=jenkins-hbase4.apache.org,40991,1683302023016 in 224 msec 2023-05-05 15:55:32,078 INFO [StoreOpener-e84c6af08e7e6bceea057422cc6e6cbe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e84c6af08e7e6bceea057422cc6e6cbe columnFamilyName cf1 2023-05-05 15:55:32,079 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=9b5dcde66b10fccd0c9178f6827394e9, ASSIGN in 383 msec 2023-05-05 15:55:32,089 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9b5dcde66b10fccd0c9178f6827394e9#cf1#compaction#142 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:55:32,092 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] throttle.PressureAwareThroughputController(145): 9b5dcde66b10fccd0c9178f6827394e9#cf2#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:55:32,092 DEBUG [StoreOpener-e84c6af08e7e6bceea057422cc6e6cbe-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/0a830604edc34189ae5830c6f54495a4.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/0a830604edc34189ae5830c6f54495a4-bottom 2023-05-05 15:55:32,098 DEBUG [StoreOpener-e84c6af08e7e6bceea057422cc6e6cbe-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/0af86f1ef5df40078761b720c0239432.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/0af86f1ef5df40078761b720c0239432-bottom 2023-05-05 15:55:32,102 DEBUG [StoreOpener-e84c6af08e7e6bceea057422cc6e6cbe-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/21ca3c427d50442da6f1361f29796bb7.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/21ca3c427d50442da6f1361f29796bb7-bottom 2023-05-05 15:55:32,105 DEBUG [StoreOpener-e84c6af08e7e6bceea057422cc6e6cbe-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/363d4c71262f41128d69f06264c2c9dc.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/363d4c71262f41128d69f06264c2c9dc-bottom 2023-05-05 15:55:32,109 DEBUG [StoreOpener-e84c6af08e7e6bceea057422cc6e6cbe-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/65bb1d6cc3f2438fa61cda724ebd2b75.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/65bb1d6cc3f2438fa61cda724ebd2b75-bottom 2023-05-05 15:55:32,118 DEBUG [StoreOpener-e84c6af08e7e6bceea057422cc6e6cbe-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/86e9b67593734675a3e910cdbc0caf13.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/86e9b67593734675a3e910cdbc0caf13-bottom 2023-05-05 15:55:32,120 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/.tmp/cf1/02d25f5fd87041ccaebe4928963a5757 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/02d25f5fd87041ccaebe4928963a5757 2023-05-05 15:55:32,122 DEBUG [StoreOpener-e84c6af08e7e6bceea057422cc6e6cbe-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/90b8eff45a534b20af611eff0bf47cff.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/90b8eff45a534b20af611eff0bf47cff-bottom 2023-05-05 15:55:32,122 INFO [StoreOpener-e84c6af08e7e6bceea057422cc6e6cbe-1] regionserver.HStore(310): Store=e84c6af08e7e6bceea057422cc6e6cbe/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:55:32,122 INFO [StoreOpener-e84c6af08e7e6bceea057422cc6e6cbe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e84c6af08e7e6bceea057422cc6e6cbe 2023-05-05 15:55:32,123 DEBUG [StoreOpener-e84c6af08e7e6bceea057422cc6e6cbe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2 2023-05-05 15:55:32,123 DEBUG [StoreOpener-e84c6af08e7e6bceea057422cc6e6cbe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2 2023-05-05 15:55:32,124 INFO [StoreOpener-e84c6af08e7e6bceea057422cc6e6cbe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e84c6af08e7e6bceea057422cc6e6cbe columnFamilyName cf2 2023-05-05 15:55:32,124 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/.tmp/cf2/615e749a0c1541b78fb020cd540f7be4 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/615e749a0c1541b78fb020cd540f7be4 2023-05-05 15:55:32,125 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 9b5dcde66b10fccd0c9178f6827394e9/cf1 of 9b5dcde66b10fccd0c9178f6827394e9 into 02d25f5fd87041ccaebe4928963a5757(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:55:32,125 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 9b5dcde66b10fccd0c9178f6827394e9: 2023-05-05 15:55:32,125 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9., storeName=9b5dcde66b10fccd0c9178f6827394e9/cf1, priority=9, startTime=1683302132071; duration=0sec 2023-05-05 15:55:32,125 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:55:32,128 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 9b5dcde66b10fccd0c9178f6827394e9/cf2 of 9b5dcde66b10fccd0c9178f6827394e9 into 615e749a0c1541b78fb020cd540f7be4(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:55:32,128 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 9b5dcde66b10fccd0c9178f6827394e9: 2023-05-05 15:55:32,128 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9., storeName=9b5dcde66b10fccd0c9178f6827394e9/cf2, priority=9, startTime=1683302132073; duration=0sec 2023-05-05 15:55:32,128 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:55:32,134 DEBUG [StoreOpener-e84c6af08e7e6bceea057422cc6e6cbe-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/3acc3bde4ad64554b6f92e4994b2ea0d.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/3acc3bde4ad64554b6f92e4994b2ea0d-bottom 2023-05-05 15:55:32,137 DEBUG [StoreOpener-e84c6af08e7e6bceea057422cc6e6cbe-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/4d6df678fe1543a9b597b5b483c413b9.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/4d6df678fe1543a9b597b5b483c413b9-bottom 2023-05-05 15:55:32,140 DEBUG [StoreOpener-e84c6af08e7e6bceea057422cc6e6cbe-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/587768fab2c34b5b891d6cdd8e5d2497.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/587768fab2c34b5b891d6cdd8e5d2497-bottom 2023-05-05 15:55:32,147 DEBUG [StoreOpener-e84c6af08e7e6bceea057422cc6e6cbe-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/78278f2c5a9b417da69bad0e2ad993a5.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/78278f2c5a9b417da69bad0e2ad993a5-bottom 2023-05-05 15:55:32,150 DEBUG [StoreOpener-e84c6af08e7e6bceea057422cc6e6cbe-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/8a63f8737720490395ee63cac7fd38b1.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/8a63f8737720490395ee63cac7fd38b1-bottom 2023-05-05 15:55:32,153 DEBUG [StoreOpener-e84c6af08e7e6bceea057422cc6e6cbe-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/b34e6f3dba14493cac9ced4f4566b2b7.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/b34e6f3dba14493cac9ced4f4566b2b7-bottom 2023-05-05 15:55:32,157 DEBUG [StoreOpener-e84c6af08e7e6bceea057422cc6e6cbe-1] regionserver.HStore(539): loaded hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/fd12d1c07c8d41ff83a744ea02f34eb5.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/fd12d1c07c8d41ff83a744ea02f34eb5-bottom 2023-05-05 15:55:32,157 INFO [StoreOpener-e84c6af08e7e6bceea057422cc6e6cbe-1] regionserver.HStore(310): Store=e84c6af08e7e6bceea057422cc6e6cbe/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-05 15:55:32,157 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe 2023-05-05 15:55:32,158 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe 2023-05-05 15:55:32,160 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-05 15:55:32,160 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e84c6af08e7e6bceea057422cc6e6cbe 2023-05-05 15:55:32,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e84c6af08e7e6bceea057422cc6e6cbe; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66230412, jitterRate=-0.013089954853057861}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-05 15:55:32,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e84c6af08e7e6bceea057422cc6e6cbe: 2023-05-05 15:55:32,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe., pid=149, masterSystemTime=1683302132001 2023-05-05 15:55:32,162 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:55:32,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-05 15:55:32,163 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-05 15:55:32,163 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-05 15:55:32,163 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe. 2023-05-05 15:55:32,163 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe. 2023-05-05 15:55:32,163 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1912): e84c6af08e7e6bceea057422cc6e6cbe/cf1 is initiating minor compaction (all files) 2023-05-05 15:55:32,163 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1912): e84c6af08e7e6bceea057422cc6e6cbe/cf2 is initiating minor compaction (all files) 2023-05-05 15:55:32,163 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of e84c6af08e7e6bceea057422cc6e6cbe/cf1 in testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe. 2023-05-05 15:55:32,163 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegion(2259): Starting compaction of e84c6af08e7e6bceea057422cc6e6cbe/cf2 in testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe. 2023-05-05 15:55:32,164 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/4d6df678fe1543a9b597b5b483c413b9.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/4d6df678fe1543a9b597b5b483c413b9-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/587768fab2c34b5b891d6cdd8e5d2497.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/587768fab2c34b5b891d6cdd8e5d2497-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/b34e6f3dba14493cac9ced4f4566b2b7.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/b34e6f3dba14493cac9ced4f4566b2b7-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/3acc3bde4ad64554b6f92e4994b2ea0d.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/3acc3bde4ad64554b6f92e4994b2ea0d-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/8a63f8737720490395ee63cac7fd38b1.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/8a63f8737720490395ee63cac7fd38b1-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/fd12d1c07c8d41ff83a744ea02f34eb5.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/fd12d1c07c8d41ff83a744ea02f34eb5-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/78278f2c5a9b417da69bad0e2ad993a5.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/78278f2c5a9b417da69bad0e2ad993a5-bottom] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/.tmp, totalSize=34.9 K 2023-05-05 15:55:32,164 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/363d4c71262f41128d69f06264c2c9dc.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/363d4c71262f41128d69f06264c2c9dc-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/86e9b67593734675a3e910cdbc0caf13.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/86e9b67593734675a3e910cdbc0caf13-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/21ca3c427d50442da6f1361f29796bb7.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/21ca3c427d50442da6f1361f29796bb7-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/0a830604edc34189ae5830c6f54495a4.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/0a830604edc34189ae5830c6f54495a4-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/0af86f1ef5df40078761b720c0239432.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/0af86f1ef5df40078761b720c0239432-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/65bb1d6cc3f2438fa61cda724ebd2b75.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/65bb1d6cc3f2438fa61cda724ebd2b75-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/90b8eff45a534b20af611eff0bf47cff.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/90b8eff45a534b20af611eff0bf47cff-bottom] into tmpdir=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/.tmp, totalSize=34.9 K 2023-05-05 15:55:32,164 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 4d6df678fe1543a9b597b5b483c413b9.53274b7324db593bcdb14d5b85f1c6b7, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683302129389 2023-05-05 15:55:32,164 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 363d4c71262f41128d69f06264c2c9dc.53274b7324db593bcdb14d5b85f1c6b7, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683302129389 2023-05-05 15:55:32,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe. 2023-05-05 15:55:32,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe. 2023-05-05 15:55:32,165 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 587768fab2c34b5b891d6cdd8e5d2497.53274b7324db593bcdb14d5b85f1c6b7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683302129557 2023-05-05 15:55:32,166 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 86e9b67593734675a3e910cdbc0caf13.53274b7324db593bcdb14d5b85f1c6b7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683302129557 2023-05-05 15:55:32,166 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=e84c6af08e7e6bceea057422cc6e6cbe, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:32,166 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting b34e6f3dba14493cac9ced4f4566b2b7.53274b7324db593bcdb14d5b85f1c6b7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683302129733 2023-05-05 15:55:32,166 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683302132166"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683302132166"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683302132166"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683302132166"}]},"ts":"1683302132166"} 2023-05-05 15:55:32,166 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 21ca3c427d50442da6f1361f29796bb7.53274b7324db593bcdb14d5b85f1c6b7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683302129733 2023-05-05 15:55:32,166 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 3acc3bde4ad64554b6f92e4994b2ea0d.53274b7324db593bcdb14d5b85f1c6b7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683302130304 2023-05-05 15:55:32,166 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 0a830604edc34189ae5830c6f54495a4.53274b7324db593bcdb14d5b85f1c6b7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683302130304 2023-05-05 15:55:32,167 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 8a63f8737720490395ee63cac7fd38b1.53274b7324db593bcdb14d5b85f1c6b7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683302130486 2023-05-05 15:55:32,167 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 0af86f1ef5df40078761b720c0239432.53274b7324db593bcdb14d5b85f1c6b7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683302130486 2023-05-05 15:55:32,167 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting fd12d1c07c8d41ff83a744ea02f34eb5.53274b7324db593bcdb14d5b85f1c6b7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683302130662 2023-05-05 15:55:32,167 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 65bb1d6cc3f2438fa61cda724ebd2b75.53274b7324db593bcdb14d5b85f1c6b7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683302130662 2023-05-05 15:55:32,167 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] compactions.Compactor(207): Compacting 78278f2c5a9b417da69bad0e2ad993a5.53274b7324db593bcdb14d5b85f1c6b7, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683302130833 2023-05-05 15:55:32,168 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] compactions.Compactor(207): Compacting 90b8eff45a534b20af611eff0bf47cff.53274b7324db593bcdb14d5b85f1c6b7, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683302130833 2023-05-05 15:55:32,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=147 2023-05-05 15:55:32,169 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=147, state=SUCCESS; OpenRegionProcedure e84c6af08e7e6bceea057422cc6e6cbe, server=jenkins-hbase4.apache.org,40991,1683302023016 in 318 msec 2023-05-05 15:55:32,170 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=147, resume processing ppid=144 2023-05-05 15:55:32,170 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=e84c6af08e7e6bceea057422cc6e6cbe, ASSIGN in 474 msec 2023-05-05 15:55:32,171 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=53274b7324db593bcdb14d5b85f1c6b7, daughterA=e84c6af08e7e6bceea057422cc6e6cbe, daughterB=9b5dcde66b10fccd0c9178f6827394e9 in 1.3300 sec 2023-05-05 15:55:32,176 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] throttle.PressureAwareThroughputController(145): e84c6af08e7e6bceea057422cc6e6cbe#cf1#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:55:32,177 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] throttle.PressureAwareThroughputController(145): e84c6af08e7e6bceea057422cc6e6cbe#cf2#compaction#145 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-05 15:55:32,190 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/.tmp/cf1/6b139889a8af447591e7b72475fbe990 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/6b139889a8af447591e7b72475fbe990 2023-05-05 15:55:32,190 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/.tmp/cf2/ec2b51d7df224cc1a450fbe9ac47e246 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/ec2b51d7df224cc1a450fbe9ac47e246 2023-05-05 15:55:32,194 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e84c6af08e7e6bceea057422cc6e6cbe/cf2 of e84c6af08e7e6bceea057422cc6e6cbe into ec2b51d7df224cc1a450fbe9ac47e246(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:55:32,194 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e84c6af08e7e6bceea057422cc6e6cbe/cf1 of e84c6af08e7e6bceea057422cc6e6cbe into 6b139889a8af447591e7b72475fbe990(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-05 15:55:32,194 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for e84c6af08e7e6bceea057422cc6e6cbe: 2023-05-05 15:55:32,194 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for e84c6af08e7e6bceea057422cc6e6cbe: 2023-05-05 15:55:32,194 INFO [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe., storeName=e84c6af08e7e6bceea057422cc6e6cbe/cf2, priority=9, startTime=1683302132163; duration=0sec 2023-05-05 15:55:32,194 INFO [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe., storeName=e84c6af08e7e6bceea057422cc6e6cbe/cf1, priority=9, startTime=1683302132162; duration=0sec 2023-05-05 15:55:32,194 DEBUG [RS:2;jenkins-hbase4:40991-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:55:32,194 DEBUG [RS:2;jenkins-hbase4:40991-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-05 15:55:32,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40991] ipc.CallRunner(144): callId: 1120 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:60952 deadline: 1683302192342, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7. is not online on jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:32,452 DEBUG [Listener at localhost/35117] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-05 15:55:32,458 INFO [Listener at localhost/35117] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-05-05 15:55:32,459 DEBUG [Listener at localhost/35117] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-05 15:55:32,467 WARN [Listener at localhost/35117] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-05 15:55:32,469 INFO [Listener at localhost/35117] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-05-05 15:55:32,469 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegion 2023-05-05 15:55:32,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-05-05 15:55:32,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-05-05 15:55:32,473 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302132473"}]},"ts":"1683302132473"} 2023-05-05 15:55:32,474 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-05-05 15:55:32,475 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-05-05 15:55:32,476 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=e84c6af08e7e6bceea057422cc6e6cbe, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=9b5dcde66b10fccd0c9178f6827394e9, UNASSIGN}] 2023-05-05 15:55:32,477 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=9b5dcde66b10fccd0c9178f6827394e9, UNASSIGN 2023-05-05 15:55:32,478 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=9b5dcde66b10fccd0c9178f6827394e9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:32,478 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683302132478"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302132478"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302132478"}]},"ts":"1683302132478"} 2023-05-05 15:55:32,479 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 9b5dcde66b10fccd0c9178f6827394e9, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:55:32,481 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=e84c6af08e7e6bceea057422cc6e6cbe, UNASSIGN 2023-05-05 15:55:32,481 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=e84c6af08e7e6bceea057422cc6e6cbe, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:32,481 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683302132481"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683302132481"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683302132481"}]},"ts":"1683302132481"} 2023-05-05 15:55:32,484 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure e84c6af08e7e6bceea057422cc6e6cbe, server=jenkins-hbase4.apache.org,40991,1683302023016}] 2023-05-05 15:55:32,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-05-05 15:55:32,631 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 9b5dcde66b10fccd0c9178f6827394e9 2023-05-05 15:55:32,631 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 9b5dcde66b10fccd0c9178f6827394e9, disabling compactions & flushes 2023-05-05 15:55:32,631 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9. 2023-05-05 15:55:32,631 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9. 2023-05-05 15:55:32,631 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9. after waiting 0 ms 2023-05-05 15:55:32,631 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9. 2023-05-05 15:55:32,631 DEBUG [StoreCloser-testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/363d4c71262f41128d69f06264c2c9dc.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/363d4c71262f41128d69f06264c2c9dc-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/86e9b67593734675a3e910cdbc0caf13.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/86e9b67593734675a3e910cdbc0caf13-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/21ca3c427d50442da6f1361f29796bb7.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/21ca3c427d50442da6f1361f29796bb7-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/0a830604edc34189ae5830c6f54495a4.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/0a830604edc34189ae5830c6f54495a4-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/0af86f1ef5df40078761b720c0239432.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/0af86f1ef5df40078761b720c0239432-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/65bb1d6cc3f2438fa61cda724ebd2b75.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/65bb1d6cc3f2438fa61cda724ebd2b75-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/90b8eff45a534b20af611eff0bf47cff.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/90b8eff45a534b20af611eff0bf47cff-top] to archive 2023-05-05 15:55:32,632 DEBUG [StoreCloser-testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-05 15:55:32,634 DEBUG [StoreCloser-testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/363d4c71262f41128d69f06264c2c9dc.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/363d4c71262f41128d69f06264c2c9dc.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,635 DEBUG [StoreCloser-testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/86e9b67593734675a3e910cdbc0caf13.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/86e9b67593734675a3e910cdbc0caf13.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,636 DEBUG [StoreCloser-testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/21ca3c427d50442da6f1361f29796bb7.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/21ca3c427d50442da6f1361f29796bb7.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,637 DEBUG [StoreCloser-testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/0a830604edc34189ae5830c6f54495a4.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/0a830604edc34189ae5830c6f54495a4.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,638 DEBUG [StoreCloser-testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/0af86f1ef5df40078761b720c0239432.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/0af86f1ef5df40078761b720c0239432.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,638 DEBUG [StoreCloser-testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/65bb1d6cc3f2438fa61cda724ebd2b75.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/65bb1d6cc3f2438fa61cda724ebd2b75.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,639 DEBUG [StoreCloser-testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/90b8eff45a534b20af611eff0bf47cff.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/90b8eff45a534b20af611eff0bf47cff.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,641 DEBUG [StoreCloser-testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/4d6df678fe1543a9b597b5b483c413b9.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/4d6df678fe1543a9b597b5b483c413b9-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/587768fab2c34b5b891d6cdd8e5d2497.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/587768fab2c34b5b891d6cdd8e5d2497-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/b34e6f3dba14493cac9ced4f4566b2b7.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/b34e6f3dba14493cac9ced4f4566b2b7-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/3acc3bde4ad64554b6f92e4994b2ea0d.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/3acc3bde4ad64554b6f92e4994b2ea0d-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/8a63f8737720490395ee63cac7fd38b1.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/8a63f8737720490395ee63cac7fd38b1-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/fd12d1c07c8d41ff83a744ea02f34eb5.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/fd12d1c07c8d41ff83a744ea02f34eb5-top, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/78278f2c5a9b417da69bad0e2ad993a5.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/78278f2c5a9b417da69bad0e2ad993a5-top] to archive 2023-05-05 15:55:32,641 DEBUG [StoreCloser-testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-05 15:55:32,642 DEBUG [StoreCloser-testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/4d6df678fe1543a9b597b5b483c413b9.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/4d6df678fe1543a9b597b5b483c413b9.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,643 DEBUG [StoreCloser-testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/587768fab2c34b5b891d6cdd8e5d2497.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/587768fab2c34b5b891d6cdd8e5d2497.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,644 DEBUG [StoreCloser-testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/b34e6f3dba14493cac9ced4f4566b2b7.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/b34e6f3dba14493cac9ced4f4566b2b7.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,645 DEBUG [StoreCloser-testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/3acc3bde4ad64554b6f92e4994b2ea0d.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/3acc3bde4ad64554b6f92e4994b2ea0d.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,646 DEBUG [StoreCloser-testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/8a63f8737720490395ee63cac7fd38b1.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/8a63f8737720490395ee63cac7fd38b1.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,647 DEBUG [StoreCloser-testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/fd12d1c07c8d41ff83a744ea02f34eb5.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/fd12d1c07c8d41ff83a744ea02f34eb5.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,648 DEBUG [StoreCloser-testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/78278f2c5a9b417da69bad0e2ad993a5.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/78278f2c5a9b417da69bad0e2ad993a5.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-05 15:55:32,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:55:32,651 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9. 2023-05-05 15:55:32,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 9b5dcde66b10fccd0c9178f6827394e9: 2023-05-05 15:55:32,653 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 9b5dcde66b10fccd0c9178f6827394e9 2023-05-05 15:55:32,653 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close e84c6af08e7e6bceea057422cc6e6cbe 2023-05-05 15:55:32,653 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e84c6af08e7e6bceea057422cc6e6cbe, disabling compactions & flushes 2023-05-05 15:55:32,653 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe. 2023-05-05 15:55:32,653 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe. 2023-05-05 15:55:32,653 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe. after waiting 0 ms 2023-05-05 15:55:32,653 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe. 2023-05-05 15:55:32,653 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=9b5dcde66b10fccd0c9178f6827394e9, regionState=CLOSED 2023-05-05 15:55:32,653 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683302132653"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302132653"}]},"ts":"1683302132653"} 2023-05-05 15:55:32,654 DEBUG [StoreCloser-testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/363d4c71262f41128d69f06264c2c9dc.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/363d4c71262f41128d69f06264c2c9dc-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/86e9b67593734675a3e910cdbc0caf13.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/86e9b67593734675a3e910cdbc0caf13-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/21ca3c427d50442da6f1361f29796bb7.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/21ca3c427d50442da6f1361f29796bb7-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/0a830604edc34189ae5830c6f54495a4.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/0a830604edc34189ae5830c6f54495a4-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/0af86f1ef5df40078761b720c0239432.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/0af86f1ef5df40078761b720c0239432-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/65bb1d6cc3f2438fa61cda724ebd2b75.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/65bb1d6cc3f2438fa61cda724ebd2b75-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/90b8eff45a534b20af611eff0bf47cff.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/90b8eff45a534b20af611eff0bf47cff-bottom] to archive 2023-05-05 15:55:32,654 DEBUG [StoreCloser-testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-05 15:55:32,655 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-05-05 15:55:32,655 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 9b5dcde66b10fccd0c9178f6827394e9, server=jenkins-hbase4.apache.org,40991,1683302023016 in 175 msec 2023-05-05 15:55:32,656 DEBUG [StoreCloser-testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/363d4c71262f41128d69f06264c2c9dc.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/363d4c71262f41128d69f06264c2c9dc.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,656 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=9b5dcde66b10fccd0c9178f6827394e9, UNASSIGN in 180 msec 2023-05-05 15:55:32,657 DEBUG [StoreCloser-testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/86e9b67593734675a3e910cdbc0caf13.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/86e9b67593734675a3e910cdbc0caf13.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,658 DEBUG [StoreCloser-testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/21ca3c427d50442da6f1361f29796bb7.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/21ca3c427d50442da6f1361f29796bb7.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,659 DEBUG [StoreCloser-testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/0a830604edc34189ae5830c6f54495a4.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/0a830604edc34189ae5830c6f54495a4.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,660 DEBUG [StoreCloser-testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/0af86f1ef5df40078761b720c0239432.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/0af86f1ef5df40078761b720c0239432.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,660 DEBUG [StoreCloser-testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/65bb1d6cc3f2438fa61cda724ebd2b75.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/65bb1d6cc3f2438fa61cda724ebd2b75.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,661 DEBUG [StoreCloser-testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/90b8eff45a534b20af611eff0bf47cff.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/90b8eff45a534b20af611eff0bf47cff.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,663 DEBUG [StoreCloser-testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/4d6df678fe1543a9b597b5b483c413b9.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/4d6df678fe1543a9b597b5b483c413b9-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/587768fab2c34b5b891d6cdd8e5d2497.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/587768fab2c34b5b891d6cdd8e5d2497-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/b34e6f3dba14493cac9ced4f4566b2b7.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/b34e6f3dba14493cac9ced4f4566b2b7-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/3acc3bde4ad64554b6f92e4994b2ea0d.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/3acc3bde4ad64554b6f92e4994b2ea0d-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/8a63f8737720490395ee63cac7fd38b1.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/8a63f8737720490395ee63cac7fd38b1-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/fd12d1c07c8d41ff83a744ea02f34eb5.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/fd12d1c07c8d41ff83a744ea02f34eb5-bottom, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/78278f2c5a9b417da69bad0e2ad993a5.53274b7324db593bcdb14d5b85f1c6b7->hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/78278f2c5a9b417da69bad0e2ad993a5-bottom] to archive 2023-05-05 15:55:32,663 DEBUG [StoreCloser-testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-05 15:55:32,664 DEBUG [StoreCloser-testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/4d6df678fe1543a9b597b5b483c413b9.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/4d6df678fe1543a9b597b5b483c413b9.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,665 DEBUG [StoreCloser-testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/587768fab2c34b5b891d6cdd8e5d2497.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/587768fab2c34b5b891d6cdd8e5d2497.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,666 DEBUG [StoreCloser-testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/b34e6f3dba14493cac9ced4f4566b2b7.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/b34e6f3dba14493cac9ced4f4566b2b7.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,667 DEBUG [StoreCloser-testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/3acc3bde4ad64554b6f92e4994b2ea0d.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/3acc3bde4ad64554b6f92e4994b2ea0d.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,668 DEBUG [StoreCloser-testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/8a63f8737720490395ee63cac7fd38b1.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/8a63f8737720490395ee63cac7fd38b1.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,669 DEBUG [StoreCloser-testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/fd12d1c07c8d41ff83a744ea02f34eb5.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/fd12d1c07c8d41ff83a744ea02f34eb5.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,670 DEBUG [StoreCloser-testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/78278f2c5a9b417da69bad0e2ad993a5.53274b7324db593bcdb14d5b85f1c6b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/78278f2c5a9b417da69bad0e2ad993a5.53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,672 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-05 15:55:32,673 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:55:32,673 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe. 2023-05-05 15:55:32,673 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e84c6af08e7e6bceea057422cc6e6cbe: 2023-05-05 15:55:32,674 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed e84c6af08e7e6bceea057422cc6e6cbe 2023-05-05 15:55:32,674 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=e84c6af08e7e6bceea057422cc6e6cbe, regionState=CLOSED 2023-05-05 15:55:32,674 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683302132674"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683302132674"}]},"ts":"1683302132674"} 2023-05-05 15:55:32,676 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-05-05 15:55:32,676 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure e84c6af08e7e6bceea057422cc6e6cbe, server=jenkins-hbase4.apache.org,40991,1683302023016 in 193 msec 2023-05-05 15:55:32,677 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=152, resume processing ppid=151 2023-05-05 15:55:32,677 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=e84c6af08e7e6bceea057422cc6e6cbe, UNASSIGN in 200 msec 2023-05-05 15:55:32,678 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683302132678"}]},"ts":"1683302132678"} 2023-05-05 15:55:32,679 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-05-05 15:55:32,680 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-05-05 15:55:32,681 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 211 msec 2023-05-05 15:55:32,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-05-05 15:55:32,775 INFO [Listener at localhost/35117] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-05-05 15:55:32,775 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegion 2023-05-05 15:55:32,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-05-05 15:55:32,778 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-05-05 15:55:32,778 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-05-05 15:55:32,779 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-05-05 15:55:32,782 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,782 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9 2023-05-05 15:55:32,782 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe 2023-05-05 15:55:32,784 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/recovered.edits] 2023-05-05 15:55:32,785 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/recovered.edits] 2023-05-05 15:55:32,785 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2, FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/recovered.edits] 2023-05-05 15:55:32,788 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/02d25f5fd87041ccaebe4928963a5757 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf1/02d25f5fd87041ccaebe4928963a5757 2023-05-05 15:55:32,788 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/6b139889a8af447591e7b72475fbe990 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf1/6b139889a8af447591e7b72475fbe990 2023-05-05 15:55:32,789 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/0a830604edc34189ae5830c6f54495a4 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/0a830604edc34189ae5830c6f54495a4 2023-05-05 15:55:32,790 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/0af86f1ef5df40078761b720c0239432 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/0af86f1ef5df40078761b720c0239432 2023-05-05 15:55:32,790 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/615e749a0c1541b78fb020cd540f7be4 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/cf2/615e749a0c1541b78fb020cd540f7be4 2023-05-05 15:55:32,791 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/ec2b51d7df224cc1a450fbe9ac47e246 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/cf2/ec2b51d7df224cc1a450fbe9ac47e246 2023-05-05 15:55:32,791 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/21ca3c427d50442da6f1361f29796bb7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/21ca3c427d50442da6f1361f29796bb7 2023-05-05 15:55:32,792 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/363d4c71262f41128d69f06264c2c9dc to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/363d4c71262f41128d69f06264c2c9dc 2023-05-05 15:55:32,793 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/recovered.edits/90.seqid to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9/recovered.edits/90.seqid 2023-05-05 15:55:32,793 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/65bb1d6cc3f2438fa61cda724ebd2b75 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/65bb1d6cc3f2438fa61cda724ebd2b75 2023-05-05 15:55:32,794 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/recovered.edits/90.seqid to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe/recovered.edits/90.seqid 2023-05-05 15:55:32,794 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/9b5dcde66b10fccd0c9178f6827394e9 2023-05-05 15:55:32,794 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/e84c6af08e7e6bceea057422cc6e6cbe 2023-05-05 15:55:32,795 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/86e9b67593734675a3e910cdbc0caf13 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/86e9b67593734675a3e910cdbc0caf13 2023-05-05 15:55:32,795 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/90b8eff45a534b20af611eff0bf47cff to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf1/90b8eff45a534b20af611eff0bf47cff 2023-05-05 15:55:32,797 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/3acc3bde4ad64554b6f92e4994b2ea0d to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/3acc3bde4ad64554b6f92e4994b2ea0d 2023-05-05 15:55:32,798 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/4d6df678fe1543a9b597b5b483c413b9 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/4d6df678fe1543a9b597b5b483c413b9 2023-05-05 15:55:32,799 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/587768fab2c34b5b891d6cdd8e5d2497 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/587768fab2c34b5b891d6cdd8e5d2497 2023-05-05 15:55:32,800 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/78278f2c5a9b417da69bad0e2ad993a5 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/78278f2c5a9b417da69bad0e2ad993a5 2023-05-05 15:55:32,801 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/8a63f8737720490395ee63cac7fd38b1 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/8a63f8737720490395ee63cac7fd38b1 2023-05-05 15:55:32,802 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/b34e6f3dba14493cac9ced4f4566b2b7 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/b34e6f3dba14493cac9ced4f4566b2b7 2023-05-05 15:55:32,803 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/fd12d1c07c8d41ff83a744ea02f34eb5 to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/cf2/fd12d1c07c8d41ff83a744ea02f34eb5 2023-05-05 15:55:32,805 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/recovered.edits/85.seqid to hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/archive/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7/recovered.edits/85.seqid 2023-05-05 15:55:32,805 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/.tmp/data/default/testSplitTableRegion/53274b7324db593bcdb14d5b85f1c6b7 2023-05-05 15:55:32,805 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-05-05 15:55:32,807 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-05-05 15:55:32,808 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-05-05 15:55:32,810 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-05-05 15:55:32,811 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-05-05 15:55:32,811 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-05-05 15:55:32,811 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302132811"}]},"ts":"9223372036854775807"} 2023-05-05 15:55:32,811 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302132811"}]},"ts":"9223372036854775807"} 2023-05-05 15:55:32,811 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683302132811"}]},"ts":"9223372036854775807"} 2023-05-05 15:55:32,813 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-05 15:55:32,813 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 53274b7324db593bcdb14d5b85f1c6b7, NAME => 'testSplitTableRegion,,1683302128885.53274b7324db593bcdb14d5b85f1c6b7.', STARTKEY => '', ENDKEY => ''}, {ENCODED => e84c6af08e7e6bceea057422cc6e6cbe, NAME => 'testSplitTableRegion,,1683302130840.e84c6af08e7e6bceea057422cc6e6cbe.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 9b5dcde66b10fccd0c9178f6827394e9, NAME => 'testSplitTableRegion,41,1683302130840.9b5dcde66b10fccd0c9178f6827394e9.', STARTKEY => '41', ENDKEY => ''}] 2023-05-05 15:55:32,813 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-05-05 15:55:32,813 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683302132813"}]},"ts":"9223372036854775807"} 2023-05-05 15:55:32,814 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-05-05 15:55:32,815 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-05-05 15:55:32,816 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 40 msec 2023-05-05 15:55:32,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-05-05 15:55:32,880 INFO [Listener at localhost/35117] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-05-05 15:55:32,894 INFO [Listener at localhost/35117] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=377 (was 377), OpenFileDescriptor=859 (was 859), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=261 (was 261), ProcessCount=179 (was 179), AvailableMemoryMB=5097 (was 5106) 2023-05-05 15:55:32,895 INFO [Listener at localhost/35117] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-05-05 15:55:32,895 INFO [Listener at localhost/35117] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-05-05 15:55:32,895 DEBUG [Listener at localhost/35117] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x745e3448 to 127.0.0.1:51118 2023-05-05 15:55:32,895 DEBUG [Listener at localhost/35117] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-05 15:55:32,896 DEBUG [Listener at localhost/35117] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-05-05 15:55:32,896 DEBUG [Listener at localhost/35117] util.JVMClusterUtil(257): Found active master hash=1571646781, stopped=false 2023-05-05 15:55:32,896 INFO [Listener at localhost/35117] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,44213,1683302021756 2023-05-05 15:55:32,903 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-05-05 15:55:32,903 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-05-05 15:55:32,903 INFO [Listener at localhost/35117] procedure2.ProcedureExecutor(629): Stopping 2023-05-05 15:55:32,903 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-05-05 15:55:32,903 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-05 15:55:32,903 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-05 15:55:32,903 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-05-05 15:55:32,903 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-05 15:55:32,904 DEBUG [Listener at localhost/35117] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5f52dad7 to 127.0.0.1:51118 2023-05-05 15:55:32,904 DEBUG [Listener at localhost/35117] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-05 15:55:32,904 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-05 15:55:32,904 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-05 15:55:32,904 INFO [Listener at localhost/35117] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,43023,1683302022922' ***** 2023-05-05 15:55:32,904 INFO [Listener at localhost/35117] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-05-05 15:55:32,904 INFO [Listener at localhost/35117] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,42799,1683302022977' ***** 2023-05-05 15:55:32,905 INFO [Listener at localhost/35117] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-05-05 15:55:32,905 INFO [RS:0;jenkins-hbase4:43023] regionserver.HeapMemoryManager(220): Stopping 2023-05-05 15:55:32,905 INFO [Listener at localhost/35117] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,40991,1683302023016' ***** 2023-05-05 15:55:32,905 INFO [Listener at localhost/35117] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-05-05 15:55:32,905 INFO [RS:0;jenkins-hbase4:43023] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-05-05 15:55:32,905 INFO [RS:2;jenkins-hbase4:40991] regionserver.HeapMemoryManager(220): Stopping 2023-05-05 15:55:32,905 INFO [RS:0;jenkins-hbase4:43023] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-05-05 15:55:32,905 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-05-05 15:55:32,905 INFO [RS:1;jenkins-hbase4:42799] regionserver.HeapMemoryManager(220): Stopping 2023-05-05 15:55:32,905 INFO [RS:2;jenkins-hbase4:40991] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-05-05 15:55:32,905 INFO [RS:1;jenkins-hbase4:42799] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-05-05 15:55:32,905 INFO [RS:2;jenkins-hbase4:40991] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-05-05 15:55:32,905 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-05-05 15:55:32,905 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-05-05 15:55:32,905 INFO [RS:0;jenkins-hbase4:43023] regionserver.HRegionServer(3304): Received CLOSE for d05f6ba26fbb949a8331f3129bf8461a 2023-05-05 15:55:32,905 INFO [RS:2;jenkins-hbase4:40991] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:32,905 INFO [RS:1;jenkins-hbase4:42799] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-05-05 15:55:32,906 DEBUG [RS:2;jenkins-hbase4:40991] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x21a06c2b to 127.0.0.1:51118 2023-05-05 15:55:32,906 INFO [RS:1;jenkins-hbase4:42799] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:32,906 DEBUG [RS:1;jenkins-hbase4:42799] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x12dde343 to 127.0.0.1:51118 2023-05-05 15:55:32,906 INFO [RS:0;jenkins-hbase4:43023] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:32,906 DEBUG [RS:2;jenkins-hbase4:40991] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-05 15:55:32,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d05f6ba26fbb949a8331f3129bf8461a, disabling compactions & flushes 2023-05-05 15:55:32,906 DEBUG [RS:0;jenkins-hbase4:43023] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x16d749ca to 127.0.0.1:51118 2023-05-05 15:55:32,906 INFO [RS:2;jenkins-hbase4:40991] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-05-05 15:55:32,906 DEBUG [RS:1;jenkins-hbase4:42799] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-05 15:55:32,906 INFO [RS:2;jenkins-hbase4:40991] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-05-05 15:55:32,906 INFO [RS:1;jenkins-hbase4:42799] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,42799,1683302022977; all regions closed. 2023-05-05 15:55:32,906 DEBUG [RS:0;jenkins-hbase4:43023] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-05 15:55:32,906 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1683302024926.d05f6ba26fbb949a8331f3129bf8461a. 2023-05-05 15:55:32,906 INFO [RS:0;jenkins-hbase4:43023] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-05-05 15:55:32,906 DEBUG [RS:0;jenkins-hbase4:43023] regionserver.HRegionServer(1479): Online Regions={d05f6ba26fbb949a8331f3129bf8461a=hbase:namespace,,1683302024926.d05f6ba26fbb949a8331f3129bf8461a.} 2023-05-05 15:55:32,906 INFO [RS:2;jenkins-hbase4:40991] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-05-05 15:55:32,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1683302024926.d05f6ba26fbb949a8331f3129bf8461a. 2023-05-05 15:55:32,907 INFO [RS:2;jenkins-hbase4:40991] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-05-05 15:55:32,907 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1683302024926.d05f6ba26fbb949a8331f3129bf8461a. after waiting 0 ms 2023-05-05 15:55:32,907 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1683302024926.d05f6ba26fbb949a8331f3129bf8461a. 2023-05-05 15:55:32,907 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing d05f6ba26fbb949a8331f3129bf8461a 1/1 column families, dataSize=78 B heapSize=488 B 2023-05-05 15:55:32,907 INFO [RS:2;jenkins-hbase4:40991] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-05-05 15:55:32,907 DEBUG [RS:2;jenkins-hbase4:40991] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-05-05 15:55:32,907 DEBUG [RS:0;jenkins-hbase4:43023] regionserver.HRegionServer(1505): Waiting on d05f6ba26fbb949a8331f3129bf8461a 2023-05-05 15:55:32,907 DEBUG [RS:2;jenkins-hbase4:40991] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-05-05 15:55:32,907 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-05-05 15:55:32,908 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-05-05 15:55:32,908 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-05-05 15:55:32,908 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-05-05 15:55:32,908 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-05-05 15:55:32,908 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-05-05 15:55:32,921 DEBUG [RS:1;jenkins-hbase4:42799] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/oldWALs 2023-05-05 15:55:32,921 INFO [RS:1;jenkins-hbase4:42799] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42799%2C1683302022977:(num 1683302024471) 2023-05-05 15:55:32,921 DEBUG [RS:1;jenkins-hbase4:42799] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-05 15:55:32,921 INFO [RS:1;jenkins-hbase4:42799] regionserver.LeaseManager(133): Closed leases 2023-05-05 15:55:32,922 INFO [RS:1;jenkins-hbase4:42799] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-05-05 15:55:32,922 INFO [RS:1;jenkins-hbase4:42799] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-05-05 15:55:32,922 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-05-05 15:55:32,922 INFO [RS:1;jenkins-hbase4:42799] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-05-05 15:55:32,922 INFO [RS:1;jenkins-hbase4:42799] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-05-05 15:55:32,924 INFO [RS:1;jenkins-hbase4:42799] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42799 2023-05-05 15:55:33,106 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-05-05 15:55:33,106 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-05-05 15:55:33,106 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-05-05 15:55:33,107 DEBUG [RS:0;jenkins-hbase4:43023] regionserver.HRegionServer(1505): Waiting on d05f6ba26fbb949a8331f3129bf8461a 2023-05-05 15:55:33,108 DEBUG [RS:2;jenkins-hbase4:40991] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-05-05 15:55:33,108 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:33,108 ERROR [BP-1744589926-172.31.14.131-1683302017433 heartbeating to localhost/127.0.0.1:35855] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1744589926-172.31.14.131-1683302017433 (Datanode Uuid 8e12ece6-ff16-4724-9684-7233909ecb4c) service to localhost/127.0.0.1:35855 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-05-05 15:55:33,108 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:33,108 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-05 15:55:33,108 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-05 15:55:33,108 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42799,1683302022977 2023-05-05 15:55:33,108 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-05 15:55:33,109 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-05 15:55:33,110 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42799,1683302022977] 2023-05-05 15:55:33,110 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42799,1683302022977; numProcessing=1 2023-05-05 15:55:33,111 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/namespace/d05f6ba26fbb949a8331f3129bf8461a/.tmp/info/8bc7dba8476f40e786ec22011e06cb61 2023-05-05 15:55:33,111 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,42799,1683302022977 already deleted, retry=false 2023-05-05 15:55:33,111 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,42799,1683302022977 expired; onlineServers=2 2023-05-05 15:55:33,114 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740/.tmp/info/68624ee465ff46d2ac66390d707ee0df 2023-05-05 15:55:33,116 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/namespace/d05f6ba26fbb949a8331f3129bf8461a/.tmp/info/8bc7dba8476f40e786ec22011e06cb61 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/namespace/d05f6ba26fbb949a8331f3129bf8461a/info/8bc7dba8476f40e786ec22011e06cb61 2023-05-05 15:55:33,117 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 68624ee465ff46d2ac66390d707ee0df 2023-05-05 15:55:33,119 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/namespace/d05f6ba26fbb949a8331f3129bf8461a/info/8bc7dba8476f40e786ec22011e06cb61, entries=2, sequenceid=6, filesize=4.8 K 2023-05-05 15:55:33,121 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for d05f6ba26fbb949a8331f3129bf8461a in 214ms, sequenceid=6, compaction requested=false 2023-05-05 15:55:33,125 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/namespace/d05f6ba26fbb949a8331f3129bf8461a/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-05-05 15:55:33,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:55:33,129 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1683302024926.d05f6ba26fbb949a8331f3129bf8461a. 2023-05-05 15:55:33,129 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d05f6ba26fbb949a8331f3129bf8461a: 2023-05-05 15:55:33,129 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1683302024926.d05f6ba26fbb949a8331f3129bf8461a. 2023-05-05 15:55:33,133 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740/.tmp/rep_barrier/583f11f85b1341b591267b13d06edf57 2023-05-05 15:55:33,136 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 583f11f85b1341b591267b13d06edf57 2023-05-05 15:55:33,149 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740/.tmp/table/69a5669381bc4fe78a1f5babd951cf25 2023-05-05 15:55:33,152 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 69a5669381bc4fe78a1f5babd951cf25 2023-05-05 15:55:33,153 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740/.tmp/info/68624ee465ff46d2ac66390d707ee0df as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740/info/68624ee465ff46d2ac66390d707ee0df 2023-05-05 15:55:33,157 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 68624ee465ff46d2ac66390d707ee0df 2023-05-05 15:55:33,157 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740/info/68624ee465ff46d2ac66390d707ee0df, entries=59, sequenceid=210, filesize=11.1 K 2023-05-05 15:55:33,157 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740/.tmp/rep_barrier/583f11f85b1341b591267b13d06edf57 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740/rep_barrier/583f11f85b1341b591267b13d06edf57 2023-05-05 15:55:33,161 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 583f11f85b1341b591267b13d06edf57 2023-05-05 15:55:33,161 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740/rep_barrier/583f11f85b1341b591267b13d06edf57, entries=24, sequenceid=210, filesize=7.5 K 2023-05-05 15:55:33,162 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740/.tmp/table/69a5669381bc4fe78a1f5babd951cf25 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740/table/69a5669381bc4fe78a1f5babd951cf25 2023-05-05 15:55:33,165 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 69a5669381bc4fe78a1f5babd951cf25 2023-05-05 15:55:33,165 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740/table/69a5669381bc4fe78a1f5babd951cf25, entries=36, sequenceid=210, filesize=8.0 K 2023-05-05 15:55:33,166 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 258ms, sequenceid=210, compaction requested=false 2023-05-05 15:55:33,170 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-05-05 15:55:33,171 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-05 15:55:33,171 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-05-05 15:55:33,172 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-05-05 15:55:33,172 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-05-05 15:55:33,172 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-05-05 15:55:33,210 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-05 15:55:33,210 INFO [RS:1;jenkins-hbase4:42799] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,42799,1683302022977; zookeeper connection closed. 2023-05-05 15:55:33,210 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:42799-0x101ef8660cb0002, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-05 15:55:33,211 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@1f236f80] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@1f236f80 2023-05-05 15:55:33,263 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-05 15:55:33,308 INFO [RS:0;jenkins-hbase4:43023] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,43023,1683302022922; all regions closed. 2023-05-05 15:55:33,308 INFO [RS:2;jenkins-hbase4:40991] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,40991,1683302023016; all regions closed. 2023-05-05 15:55:33,316 DEBUG [RS:2;jenkins-hbase4:40991] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/oldWALs 2023-05-05 15:55:33,317 INFO [RS:2;jenkins-hbase4:40991] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40991%2C1683302023016.meta:.meta(num 1683302024658) 2023-05-05 15:55:33,318 DEBUG [RS:0;jenkins-hbase4:43023] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/oldWALs 2023-05-05 15:55:33,318 INFO [RS:0;jenkins-hbase4:43023] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43023%2C1683302022922:(num 1683302024471) 2023-05-05 15:55:33,318 DEBUG [RS:0;jenkins-hbase4:43023] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-05 15:55:33,318 INFO [RS:0;jenkins-hbase4:43023] regionserver.LeaseManager(133): Closed leases 2023-05-05 15:55:33,318 INFO [RS:0;jenkins-hbase4:43023] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-05-05 15:55:33,320 INFO [RS:0;jenkins-hbase4:43023] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-05-05 15:55:33,320 INFO [RS:0;jenkins-hbase4:43023] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-05-05 15:55:33,320 INFO [RS:0;jenkins-hbase4:43023] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-05-05 15:55:33,321 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-05-05 15:55:33,321 INFO [RS:0;jenkins-hbase4:43023] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43023 2023-05-05 15:55:33,324 DEBUG [RS:2;jenkins-hbase4:40991] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/oldWALs 2023-05-05 15:55:33,324 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-05 15:55:33,324 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:33,324 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43023,1683302022922 2023-05-05 15:55:33,324 INFO [RS:2;jenkins-hbase4:40991] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40991%2C1683302023016:(num 1683302024471) 2023-05-05 15:55:33,324 DEBUG [RS:2;jenkins-hbase4:40991] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-05 15:55:33,324 INFO [RS:2;jenkins-hbase4:40991] regionserver.LeaseManager(133): Closed leases 2023-05-05 15:55:33,324 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,43023,1683302022922] 2023-05-05 15:55:33,325 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43023,1683302022922; numProcessing=2 2023-05-05 15:55:33,325 INFO [RS:2;jenkins-hbase4:40991] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-05-05 15:55:33,325 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-05-05 15:55:33,325 INFO [RS:2;jenkins-hbase4:40991] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:40991 2023-05-05 15:55:33,327 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,43023,1683302022922 already deleted, retry=false 2023-05-05 15:55:33,327 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,43023,1683302022922 expired; onlineServers=1 2023-05-05 15:55:33,327 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-05 15:55:33,327 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,40991,1683302023016 2023-05-05 15:55:33,331 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,40991,1683302023016] 2023-05-05 15:55:33,331 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,40991,1683302023016; numProcessing=3 2023-05-05 15:55:33,332 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,40991,1683302023016 already deleted, retry=false 2023-05-05 15:55:33,332 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,40991,1683302023016 expired; onlineServers=0 2023-05-05 15:55:33,332 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,44213,1683302021756' ***** 2023-05-05 15:55:33,332 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-05-05 15:55:33,333 DEBUG [M:0;jenkins-hbase4:44213] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1c906706, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-05-05 15:55:33,333 INFO [M:0;jenkins-hbase4:44213] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,44213,1683302021756 2023-05-05 15:55:33,333 INFO [M:0;jenkins-hbase4:44213] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,44213,1683302021756; all regions closed. 2023-05-05 15:55:33,333 DEBUG [M:0;jenkins-hbase4:44213] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-05 15:55:33,333 DEBUG [M:0;jenkins-hbase4:44213] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-05-05 15:55:33,333 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-05-05 15:55:33,333 DEBUG [M:0;jenkins-hbase4:44213] cleaner.HFileCleaner(317): Stopping file delete threads 2023-05-05 15:55:33,333 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1683302024117] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1683302024117,5,FailOnTimeoutGroup] 2023-05-05 15:55:33,333 INFO [M:0;jenkins-hbase4:44213] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-05-05 15:55:33,333 INFO [M:0;jenkins-hbase4:44213] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-05-05 15:55:33,334 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1683302024118] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1683302024118,5,FailOnTimeoutGroup] 2023-05-05 15:55:33,334 INFO [M:0;jenkins-hbase4:44213] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-05-05 15:55:33,334 DEBUG [M:0;jenkins-hbase4:44213] master.HMaster(1512): Stopping service threads 2023-05-05 15:55:33,334 INFO [M:0;jenkins-hbase4:44213] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-05-05 15:55:33,334 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-05-05 15:55:33,334 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-05 15:55:33,334 ERROR [M:0;jenkins-hbase4:44213] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-05-05 15:55:33,334 INFO [M:0;jenkins-hbase4:44213] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-05 15:55:33,334 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-05-05 15:55:33,335 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-05-05 15:55:33,335 DEBUG [M:0;jenkins-hbase4:44213] zookeeper.ZKUtil(398): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-05-05 15:55:33,335 WARN [M:0;jenkins-hbase4:44213] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-05-05 15:55:33,335 INFO [M:0;jenkins-hbase4:44213] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-05 15:55:33,335 INFO [M:0;jenkins-hbase4:44213] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-05-05 15:55:33,335 DEBUG [M:0;jenkins-hbase4:44213] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-05-05 15:55:33,335 INFO [M:0;jenkins-hbase4:44213] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-05 15:55:33,335 DEBUG [M:0;jenkins-hbase4:44213] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-05 15:55:33,335 DEBUG [M:0;jenkins-hbase4:44213] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-05-05 15:55:33,335 DEBUG [M:0;jenkins-hbase4:44213] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-05 15:55:33,335 INFO [M:0;jenkins-hbase4:44213] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=608.13 KB heapSize=719.05 KB 2023-05-05 15:55:33,350 INFO [M:0;jenkins-hbase4:44213] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=608.13 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/04dc5b09408d42c481355b312657c7e3 2023-05-05 15:55:33,354 INFO [M:0;jenkins-hbase4:44213] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 04dc5b09408d42c481355b312657c7e3 2023-05-05 15:55:33,354 DEBUG [M:0;jenkins-hbase4:44213] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/04dc5b09408d42c481355b312657c7e3 as hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/04dc5b09408d42c481355b312657c7e3 2023-05-05 15:55:33,358 INFO [M:0;jenkins-hbase4:44213] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 04dc5b09408d42c481355b312657c7e3 2023-05-05 15:55:33,358 INFO [M:0;jenkins-hbase4:44213] regionserver.HStore(1080): Added hdfs://localhost:35855/user/jenkins/test-data/1b4df322-d3e4-ca7c-45a7-4cb3f32cbe27/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/04dc5b09408d42c481355b312657c7e3, entries=156, sequenceid=1282, filesize=34.9 K 2023-05-05 15:55:33,359 INFO [M:0;jenkins-hbase4:44213] regionserver.HRegion(2948): Finished flush of dataSize ~608.13 KB/622730, heapSize ~719.04 KB/736296, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 24ms, sequenceid=1282, compaction requested=false 2023-05-05 15:55:33,360 INFO [M:0;jenkins-hbase4:44213] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-05 15:55:33,360 DEBUG [M:0;jenkins-hbase4:44213] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-05-05 15:55:33,364 INFO [M:0;jenkins-hbase4:44213] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-05-05 15:55:33,364 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-05-05 15:55:33,365 INFO [M:0;jenkins-hbase4:44213] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:44213 2023-05-05 15:55:33,366 DEBUG [M:0;jenkins-hbase4:44213] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,44213,1683302021756 already deleted, retry=false 2023-05-05 15:55:33,426 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-05 15:55:33,426 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:43023-0x101ef8660cb0001, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-05 15:55:33,426 INFO [RS:0;jenkins-hbase4:43023] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,43023,1683302022922; zookeeper connection closed. 2023-05-05 15:55:33,427 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2462694] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2462694 2023-05-05 15:55:33,526 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-05 15:55:33,526 INFO [M:0;jenkins-hbase4:44213] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,44213,1683302021756; zookeeper connection closed. 2023-05-05 15:55:33,527 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): master:44213-0x101ef8660cb0000, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-05 15:55:33,627 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-05 15:55:33,627 DEBUG [Listener at localhost/35117-EventThread] zookeeper.ZKWatcher(600): regionserver:40991-0x101ef8660cb0003, quorum=127.0.0.1:51118, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-05 15:55:33,627 INFO [RS:2;jenkins-hbase4:40991] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,40991,1683302023016; zookeeper connection closed. 2023-05-05 15:55:33,627 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2dcc87d5] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2dcc87d5 2023-05-05 15:55:33,627 INFO [Listener at localhost/35117] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-05-05 15:55:33,628 WARN [Listener at localhost/35117] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-05-05 15:55:33,633 INFO [Listener at localhost/35117] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-05-05 15:55:33,741 WARN [BP-1744589926-172.31.14.131-1683302017433 heartbeating to localhost/127.0.0.1:35855] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-05-05 15:55:33,741 WARN [BP-1744589926-172.31.14.131-1683302017433 heartbeating to localhost/127.0.0.1:35855] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1744589926-172.31.14.131-1683302017433 (Datanode Uuid 7b0dd175-f33e-42f4-a043-9cbab024e73c) service to localhost/127.0.0.1:35855 2023-05-05 15:55:33,743 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/cluster_ed487d35-94af-afba-80b3-40ed4e353473/dfs/data/data5/current/BP-1744589926-172.31.14.131-1683302017433] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-05-05 15:55:33,744 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/cluster_ed487d35-94af-afba-80b3-40ed4e353473/dfs/data/data6/current/BP-1744589926-172.31.14.131-1683302017433] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-05-05 15:55:33,746 WARN [Listener at localhost/35117] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-05-05 15:55:33,749 INFO [Listener at localhost/35117] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-05-05 15:55:33,851 WARN [BP-1744589926-172.31.14.131-1683302017433 heartbeating to localhost/127.0.0.1:35855] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1744589926-172.31.14.131-1683302017433 (Datanode Uuid 8e12ece6-ff16-4724-9684-7233909ecb4c) service to localhost/127.0.0.1:35855 2023-05-05 15:55:33,852 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/cluster_ed487d35-94af-afba-80b3-40ed4e353473/dfs/data/data3/current/BP-1744589926-172.31.14.131-1683302017433] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-05-05 15:55:33,852 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/cluster_ed487d35-94af-afba-80b3-40ed4e353473/dfs/data/data4/current/BP-1744589926-172.31.14.131-1683302017433] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-05-05 15:55:33,854 WARN [Listener at localhost/35117] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-05-05 15:55:33,856 INFO [Listener at localhost/35117] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-05-05 15:55:33,961 WARN [BP-1744589926-172.31.14.131-1683302017433 heartbeating to localhost/127.0.0.1:35855] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-05-05 15:55:33,961 WARN [BP-1744589926-172.31.14.131-1683302017433 heartbeating to localhost/127.0.0.1:35855] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1744589926-172.31.14.131-1683302017433 (Datanode Uuid 7884a1c8-f94e-4e8e-bd24-af43573a93c6) service to localhost/127.0.0.1:35855 2023-05-05 15:55:33,962 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/cluster_ed487d35-94af-afba-80b3-40ed4e353473/dfs/data/data1/current/BP-1744589926-172.31.14.131-1683302017433] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-05-05 15:55:33,962 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7d9eb132-b92f-097e-b298-d7dbc7b73930/cluster_ed487d35-94af-afba-80b3-40ed4e353473/dfs/data/data2/current/BP-1744589926-172.31.14.131-1683302017433] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-05-05 15:55:33,985 INFO [Listener at localhost/35117] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-05-05 15:55:34,174 INFO [Listener at localhost/35117] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-05-05 15:55:34,430 INFO [Listener at localhost/35117] hbase.HBaseTestingUtility(1293): Minicluster is down